Jenkins Declarative Pipeline Syntax: 10 Gotchas That Will Burn You in Production

I remember the first time I deployed a Jenkins Pipeline to production—it was a Friday, 4:55 PM. The pipeline ran fine on my laptop, but on the production Jenkins master, it stalled for 45 minutes. Turns out, I forgot options { timestamps() } and the log was a jumbled mess. Worse, a when condition evaluated env.BRANCH_NAME as null because I hadn't set agent any at the top. That night, I learned that Declarative Pipeline is not just syntax—it's a discipline. Over the years, I've seen teams burn hours on stash size limits, matrix axis misconfigurations, and credentials leaking into logs. This article is the guide I wish I had back then: a production-tested reference for Jenkins Declarative Pipeline with every gotcha I've encountered.

1. Agent Directive: The Foundation of Your Pipeline

The agent directive tells Jenkins where to execute your pipeline or stage. In production, the most common mistake is using agent any at the top level, which randomly assigns any available agent. This can cause environment inconsistencies (e.g., different OS, missing tools). Best practice: use agent none at the pipeline level and define specific agents per stage. For example, agent { label 'linux && docker' } for a build stage and agent { label 'windows' } for a packaging stage. You can also use agent { docker 'maven:3.8.1' } to run inside a container—this ensures reproducibility. However, beware of Docker agent overhead: pulling images adds minutes. Use agent { docker { image 'maven:3.8.1' args '-v /tmp:/tmp' } } to mount volumes. Also, remember that agent can be overridden at stage level, but the stage agent inherits the pipeline agent's workspace unless you use reuseNode true. In production, I've seen pipelines fail because a stage agent tried to access files from a different node. Always stash/unstash or use shared volumes.

pipeline {
    agent none
    stages {
        stage('Build') {
            agent { label 'linux' }
            steps { echo 'Building on Linux' }
        }
        stage('Test') {
            agent { docker 'node:14' }
            steps { sh 'npm test' }
        }
    }
}

2. Stages and Steps: Structuring Your Workflow

The stages block contains one or more stage directives, each with a name and steps. Each stage runs sequentially by default. Inside steps, you use declarative steps like sh, echo, git, etc. Avoid using script blocks unless necessary—they break the declarative model and disable some features like post conditions. Use parallel within a stage for concurrency. For complex workflows, use matrix (see section 11). A common mistake is putting too many steps in one stage—split into logical stages (Build, Test, Deploy) for better visibility and error handling. Each stage can have its own agent, tools, environment, when, and post. Use stage('Name') { ... } with clear names. In production, we use stages like 'Checkout', 'Compile', 'Unit Tests', 'Integration Tests', 'Package', 'Deploy to Staging', 'Smoke Tests', 'Deploy to Prod'. This granularity helps pinpoint failures quickly.

stages {
    stage('Checkout') {
        steps { checkout scm }
    }
    stage('Build') {
        steps { sh 'make build' }
    }
    stage('Test') {
        steps { sh 'make test' }
    }
}

3. Post Conditions: Handling Success, Failure, and Always

The post block defines actions to run after a stage or pipeline completes. It supports conditions: always, success, failure, unstable, changed, fixed, regression, aborted, cleanup. Use post for cleanup (e.g., archive artifacts, send notifications, clean workspace). A common pitfall: post runs even if the stage fails, but if the agent is gone (e.g., Docker container removed), post may not execute. Use always for critical cleanup. Also, post in a stage runs after that stage; pipeline-level post runs after all stages. In production, we use post { always { cleanWs() } } to clean workspace, but be careful: cleanWs() deletes the entire workspace, which may affect parallel stages. Use post { failure { emailext ... } } to alert on failures. Note that post conditions can be combined with when for conditional notifications.

post {
    always {
        cleanWs()
    }
    success {
        echo 'Pipeline succeeded'
    }
    failure {
        emailext subject: 'Pipeline Failed', body: 'Check logs', to: 'team@example.com'
    }
}

4. Environment Variables: Managing Configuration

The environment directive allows you to define variables accessible throughout the pipeline. You can set static values, dynamic values from sh commands, or use credentials() to inject secrets. Variables declared at pipeline level are available in all stages unless overridden. Stage-level environment variables override pipeline-level ones for that stage only. Use env.VARNAME to access them in steps. A common mistake is to use environment { VAR = sh(...) } which sets VAR to the return code, not the output. Use script { env.VAR = sh(returnStdout: true, script: '...').trim() } for dynamic values. Also, beware of variable expansion: echo "$VAR" works, but sh 'echo $VAR' may not expand if the shell doesn't have the variable. Use sh "echo $VAR" with double quotes. In production, we use environment variables for build numbers, commit hashes, and artifact paths.

environment {
    APP_NAME = 'my-app'
    BUILD_NUM = "${BUILD_NUMBER}"
    CRED = credentials('my-cred')
}
stages {
    stage('Example') {
        environment {
            STAGE_VAR = 'stage-specific'
        }
        steps {
            echo "App: ${APP_NAME}, Stage: ${STAGE_VAR}"
        }
    }
}

5. Tools Directive: Maven, JDK, and More

The tools directive automatically configures tools like Maven, JDK, Gradle, etc. You must have these tools configured in Jenkins Global Tool Configuration. Use tools { maven 'Maven 3.8' } to set the PATH to include Maven. You can specify multiple tools: tools { maven 'Maven 3.8'; jdk 'JDK 11' }. Tools can be set at pipeline or stage level. A common gotcha: tool names are case-sensitive and must match exactly. Also, tools are only available on agents that have the tool installed or configured. If using Docker agents, tools may not be available unless the image has them. In production, we use Docker agents with pre-installed tools to avoid inconsistencies. Use tool name: 'Maven 3.8', type: 'maven' if you need more control. Note that tools only sets environment variables like JAVA_HOME and PATH; it doesn't install tools dynamically.

tools {
    maven 'Maven 3.8'
    jdk 'JDK 11'
}
stages {
    stage('Build') {
        steps {
            sh 'mvn clean install'
        }
    }
}

6. Triggers: Cron, Webhook, and Polling

The triggers directive defines how the pipeline is automatically triggered. Common triggers: cron (e.g., cron('0 2 *') for nightly builds), pollSCM (periodically check SCM for changes), upstream (triggered by other pipelines), and webhook (via Generic Webhook Trigger plugin). In declarative, you can only use cron, pollSCM, and upstream natively. For webhooks, you need the Generic Webhook Trigger plugin and use properties with pipelineTriggers. A common mistake: using cron without pollSCM leads to builds at fixed times regardless of changes. Use pollSCM for change-driven builds. Also, triggers only applies to the main branch by default; for PRs, use Multibranch Pipeline which automatically triggers on SCM changes. In production, we use cron for nightly integration tests and pollSCM for feature branches.

triggers {
    cron('0 2 * * *')
    pollSCM('H/15 * * * *')
}
// For webhook, use properties block:
properties([
    pipelineTriggers([
        [$class: 'GenericTrigger', ...]
    ])
])

7. Parameters: Making Pipelines Interactive

The parameters directive allows users to provide input when triggering a build. Supported types: string, text, booleanParam, choice, password, file. Define parameters at pipeline level, then access them via params.PARAM_NAME. A common mistake: using parameters in a Multibranch Pipeline—they are not supported; use properties with buildDiscarder and parameters instead. Also, parameters are evaluated at build time, so when conditions can use params. In production, we use choice for deployment environments (dev, staging, prod) and booleanParam for skipping tests. Be careful with password parameters—they are masked but can be exposed if echoed. Use credentials() for sensitive data instead.

parameters {
    string(name: 'ENVIRONMENT', defaultValue: 'staging', description: 'Target environment')
    choice(name: 'DEPLOY_STRATEGY', choices: ['rolling', 'blue-green'], description: 'Deployment strategy')
    booleanParam(name: 'RUN_TESTS', defaultValue: true, description: 'Run tests?')
}
stages {
    stage('Deploy') {
        when { expression { params.ENVIRONMENT == 'production' } }
        steps { echo "Deploying to ${params.ENVIRONMENT}" }
    }
}

8. Options: Timestamps, Timeout, Retry, and Build Discarder

The options directive configures pipeline behavior. Essential options: timestamps() (adds timestamps to console output), timeout(time: 1, unit: 'HOURS') (overall pipeline timeout), retry(3) (retry the entire pipeline on failure), buildDiscarder(logRotator(numToKeepStr: '10')) (keep only 10 builds). Other useful options: skipDefaultCheckout(), disableConcurrentBuilds(), preserveStashes(), parallelsAlwaysFailFast(). Options can be set at pipeline or stage level. A common mistake: forgetting timestamps() makes debugging impossible. Also, retry retries the whole pipeline, not individual stages—use retry inside a stage step for stage-level retries. In production, we always set timestamps(), timeout, and buildDiscarder. For long-running pipelines, we set timeout per stage as well.

options {
    timestamps()
    timeout(time: 1, unit: 'HOURS')
    retry(2)
    buildDiscarder(logRotator(numToKeepStr: '10'))
}
stages {
    stage('Test') {
        options { timeout(time: 10, unit: 'MINUTES') }
        steps { sh 'make test' }
    }
}

9. When Conditions: Branch, Expression, BuildingTag, and AnyOf

The when directive controls whether a stage executes based on conditions. Common conditions: branch 'main', expression { return env.BRANCH_NAME == 'main' }, buildingTag(), tag "v*", changeRequest(), environment name: 'VAR', value: 'val'. You can combine conditions with anyOf or allOf. A common mistake: using branch in a Multibranch Pipeline—it works, but for PRs, branch matches the source branch. Use changeRequest() for PR-specific logic. Also, when is evaluated before the stage's agent is allocated unless you set beforeAgent true. In production, we use when { branch 'main' } for deployment stages and when { buildingTag() } for release stages. Be careful with expression—it can throw exceptions if the variable is null.

when {
    anyOf {
        branch 'main'
        buildingTag()
    }
    expression { env.BRANCH_NAME != 'develop' }
    beforeAgent true
}

10. Credentials: UsernamePassword, SSH Key, and String

The credentials() helper in environment injects credentials as environment variables. Supported types: usernamePassword, sshUserPrivateKey, string, file. Use credentials('my-cred') which creates MY_CRED_USR and MY_CRED_PSW for usernamePassword. For SSH keys, it creates MY_CRED containing the private key. A common mistake: using credentials() inside a script block—it only works in environment. For dynamic credential usage, use withCredentials step. Also, never echo credentials—Jenkins masks them only if used correctly. In production, we use withCredentials([sshUserPrivateKey(credentialsId: 'ssh-key', keyFileVariable: 'SSH_KEY')]) for SSH operations. Always rotate credentials periodically.

environment {
    // This creates MY_CRED_USR and MY_CRED_PSW
    MY_CRED = credentials('my-cred')
}
stages {
    stage('Deploy') {
        steps {
            withCredentials([usernamePassword(credentialsId: 'deploy-creds', usernameVariable: 'USER', passwordVariable: 'PASS')]) {
                sh 'deploy.sh --user $USER --password $PASS'
            }
        }
    }
}

11. Matrix: Parallel Builds with Multiple Axes

The matrix directive allows you to run the same stage across multiple axis combinations (e.g., different OS, JDK versions). It's a declarative way to achieve parallelism without manual parallel blocks. Define axes like axis { name 'OS'; values 'linux', 'windows' } and axis { name 'JDK'; values '11', '17' }. Use excludes to skip invalid combinations. Each combination runs in parallel with its own agent (if specified). A common mistake: forgetting that matrix stages share the same workspace? Actually, each combination gets its own workspace. Also, matrix can be nested inside a stage. In production, we use matrix for cross-platform testing. Be careful with resource usage—too many combinations can overwhelm your Jenkins cluster. Use failFast true to stop all on first failure.

matrix {
    axes {
        axis {
            name 'OS'
            values 'linux', 'windows'
        }
        axis {
            name 'JDK'
            values '11', '17'
        }
    }
    excludes {
        exclude {
            axis {
                name 'OS'
                values 'windows'
            }
            axis {
                name 'JDK'
                values '17'
            }
        }
    }
    stages {
        stage('Test') {
            steps {
                echo "Testing on ${OS} with JDK ${JDK}"
            }
        }
    }
}

12. Script Block: The Groovy Escape Hatch (Use Sparingly)

The script block allows you to use arbitrary Groovy code inside a declarative pipeline. It's useful for complex logic that can't be expressed declaratively, like loops, conditionals, or dynamic variable assignment. However, overusing script defeats the purpose of declarative pipeline—you lose validation, error handling, and Blue Ocean visualization. Best practices: limit script blocks to a few lines, avoid side effects, and prefer declarative alternatives (e.g., when for conditions, matrix for parallelism). Common use cases: parsing JSON, generating dynamic parameters, or calling APIs. In production, we use script to read a file and set environment variables dynamically. Always wrap script in a stage and handle exceptions with try-catch.

stage('Dynamic Steps') {
    steps {
        script {
            def versions = readJSON file: 'versions.json'
            versions.each { service, version ->
                echo "Deploying ${service} version ${version}"
            }
        }
    }
}