Jenkins Pipeline: advanced automation
Jenkins remains a popular CI/CD solution for organizations that need full control over their build infrastructure. Where GitHub Actions or GitLab CI tie you to their platform, Jenkins runs on your own servers, connects to any Git repository and can be extended through a very large plugin ecosystem. That freedom comes at a price: keeping the controller, agents and plugins up to date is your job.
Since Jenkins 2, the right way to describe a build is no longer to click through the UI but to write a versioned Jenkinsfile at the root of the project. The pipeline then evolves with the code: a branch can change its own build chain, and every change goes through review like any other modification.
Declarative or scripted?
Jenkins offers two syntaxes. A scripted pipeline is free-form Groovy: powerful, but hard to read and validate. A declarative pipeline enforces a structure (pipeline, agent, stages, post) that is checked before execution. For a typical PHP application, declarative covers every need; when complex logic becomes necessary, a script { } block or a shared library lets you step outside it occasionally.
Declarative pipeline
Here is a complete pipeline for a Symfony project: dependency installation, static analysis and tests in parallel, then deployment from the main branch.
pipeline {
agent {
dockerfile {
filename 'Dockerfile.ci'
}
}
environment {
APP_ENV = 'test'
DATABASE_URL = credentials('database-url')
}
stages {
stage('Install') {
steps {
sh 'composer install --prefer-dist'
}
}
stage('Quality') {
parallel {
stage('PHPStan') {
steps {
sh 'vendor/bin/phpstan analyse src'
}
}
stage('CS Fixer') {
steps {
sh 'vendor/bin/php-cs-fixer fix --dry-run --diff'
}
}
}
}
stage('Test') {
steps {
sh 'php bin/phpunit --log-junit results.xml'
}
post {
always {
junit 'results.xml'
}
}
}
stage('Deploy') {
when {
branch 'main'
}
steps {
sh './deploy.sh production'
}
}
}
post {
failure {
slackSend channel: '#ci', message: "Build FAILED: ${env.JOB_NAME}"
}
}
}Reading the pipeline, block by block
agent { dockerfile { … } }: each build runs in a throwaway container created from the image described in the project'sDockerfile.ci(detailed below). The Docker Pipeline plugin is required, and the agent must have Docker available.environment: defines variables visible to every step.credentials('database-url')fetches a secret stored in Jenkins; for a "Secret text" credential, the variable holds the value directly and Jenkins masks it in the logs.parallel: PHPStan and PHP-CS-Fixer do not depend on each other, so they run at the same time. The parent stage fails if either one fails.junit: placed in apost { always { } }block, it publishes the test report even when tests fail. Jenkins then shows the history and flaky tests.when { branch 'main' }: this condition only works in a Multibranch Pipeline job, where Jenkins knows the branch name. In a plain Pipeline job, the stage would always be skipped.post { failure { } }: the Slack notification is only sent on failure.slackSendis provided by the Slack Notification plugin.
A build image with Composer
The official php:8.3-cli image contains neither Composer, nor git, nor the zip extension. That is why the pipeline relies on a Dockerfile.ci versioned with the project, which adds what the build needs:
FROM php:8.3-cli
RUN apt-get update \
&& apt-get install -y --no-install-recommends git unzip libzip-dev libicu-dev openssh-client \
&& docker-php-ext-install zip intl pdo_mysql \
&& rm -rf /var/lib/apt/lists/*
COPY --from=composer:2 /usr/bin/composer /usr/bin/composerWith dockerfile { filename 'Dockerfile.ci' }, Jenkins builds the image, caches it on the agent and runs the build inside it. The CI environment is thus described in the repository instead of in a machine's configuration.
A more robust pipeline
The first example works, but it lacks several safeguards that are always added in production: a maximum duration, no concurrent builds on the same branch, history rotation, artifact archiving and a human approval before going to production.
pipeline {
agent {
dockerfile {
filename 'Dockerfile.ci'
}
}
options {
timeout(time: 30, unit: 'MINUTES')
disableConcurrentBuilds()
buildDiscarder(logRotator(numToKeepStr: '20'))
timestamps()
}
environment {
APP_ENV = 'test'
COMPOSER_HOME = "${env.WORKSPACE}/.composer"
}
stages {
stage('Install') {
steps {
sh 'composer install --prefer-dist --no-progress --no-interaction'
}
}
stage('Package') {
steps {
sh 'mkdir -p dist && tar --exclude=./dist --exclude=./.git -czf dist/app.tar.gz .'
archiveArtifacts artifacts: 'dist/app.tar.gz', fingerprint: true
}
}
stage('Deploy production') {
when {
branch 'main'
beforeInput true
}
input {
message 'Deploy to production?'
ok 'Deploy'
submitter 'release-managers'
}
steps {
sshagent(credentials: ['deploy-ssh-key']) {
sh './deploy.sh production'
}
}
}
}
post {
always {
cleanWs()
}
}
}A few explanations:
timeoutaborts a stuck build (a test waiting for an unavailable service, for example) instead of tying up an executor for hours.disableConcurrentBuilds()prevents two deployments of the same branch from overlapping.COMPOSER_HOMEpoints to the workspace: Jenkins starts the container with the agent's UID, whose home directory is often not writable inside the image.inputpauses the pipeline until a member of therelease-managersgroup approves. WithbeforeInput true, the branch condition is evaluated before the prompt, so other branches are never blocked.sshagent(SSH Agent plugin) loads the deployment key only for the duration of the step, without ever writing it to the workspace.cleanWs()(Workspace Cleanup plugin) deletes the workspace at the end, so the next build starts from a clean state.
Common pitfalls
- Secret interpolation: writing
sh "mysql -p${DB_PASSWORD}"with double quotes makes Groovy interpolate the secret before execution, and Jenkins prints a security warning. Use single quotes,sh 'mysql -p"$DB_PASSWORD"', so the shell reads the environment variable itself. - Agent held during an
input: with a global agent, the executor stays reserved while waiting for approval, and the globaltimeoutkeeps running. For long waits, declareagent noneat pipeline level and one agent per stage. - Unmaintained plugins: every plugin is a dependency. Stick to the ones you need and update them regularly, as they are a frequent source of vulnerabilities.
- Builds on the controller: configure zero executors on the built-in node and run builds on dedicated agents, to protect the controller and its secrets.
Validate the Jenkinsfile before pushing
A syntax error in a Jenkinsfile is often only discovered after the push. Jenkins exposes a linter for declarative pipelines, usable with an API token:
curl -X POST --user "$JENKINS_USER:$JENKINS_TOKEN" \
-F "jenkinsfile=<Jenkinsfile" \
"$JENKINS_URL/pipeline-model-converter/validate"The response states whether the file is valid or points to the faulty line. This command fits easily into a Git hook or your editor.
Sharing code with Shared Libraries
When several projects share the same steps (notification, deployment, image publishing), duplicating the Jenkinsfile quickly becomes unmanageable. Shared Libraries let you put that common code in a separate Git repository, loaded with @Library('library-name') _ at the top of the Jenkinsfile. Each project then keeps a short, readable pipeline.
Jenkins best practices
- Use Docker agents for isolation
- Run independent stages in parallel
- Store credentials in Jenkins Credentials
- Set up notifications for failures
- Archive build artifacts
- Version the Jenkinsfile and the build image with the code
- Always define a
timeoutand history rotation
When not to choose Jenkins
If your code is hosted on GitHub or GitLab and you have no particular hosting constraint, the tools built into those platforms require far less maintenance. Jenkins really makes sense when builds must stay on your network, access internal resources, or when you already have a team able to run the platform over the long term.