Use Declarative Jenkins Pipelines

GitHub repository of exercises: https://github.com/FeynmanFan/declarative-pipelines Jenkins version used: 2.479.2 (in Docker container)

Table of Contents

1. Discover the basics of declarative pipelines

• 1.1 Introduction and context
• 1.2 What “Declarative” means in Jenkins pipelines
• 1.3 Demo: Hello, Declarative Pipeline
• 1.4 Demo: Move the pipeline to a Jenkinsfile in GitHub
• 1.5 Validation of declarative pipelines — the linter
• 1.6 Demo: Using linter on your pipeline
• 1.7 Basic Syntax Summary

2. Build multi stage and multi branch pipelines

• 2.1 Multi-stage vs Multi-branch: clarification of terms
• 2.2 Demo: Simple multi-stage pipeline with conditional stage
• 2.3 Parallelism in Jenkins pipelines
• 2.4 Demo: Simple parallelized build
• 2.5 Demo: Multi-branch pipeline
• 2.6 The role of manual steps in DevOps pipelines
• 2.7 Demo: Pipeline with manual approval step

3. Integrate declarative pipelines with build tools and notifications

• 3.1 Working with build tools in declarative pipelines
• 3.2 Demo: Build simple with the tools directive (Maven)
• 3.3 Observability in pipelines
• 3.4 Demo: Create a Slack notification for a build
• 3.5 Demo: Manage artifacts in a build
• 3.6 The external resources model
• 3.7 Demo: Trigger an Azure DevOps pipeline from Jenkins

4. Working with environment variables and secrets

• 4.1 Secrets have no place in version control
• 4.2 Demo: Working with secrets in Jenkins
• 4.3 Demo: Using environment variables in pipelines
• 4.4 Configure your pipeline
• 4.5 Demo: Pipeline with build parameters

5. Resolve errors and improve pipeline resilience

• 5.1 Error handling in declarative pipelines
• 5.2 Demo: Security cleanup with error handling
• 5.3 You don’t need Retry — but here’s how to do it
• 5.4 Demo: Retry and Timeout
• 5.5 Course summary

1. Discover the basics of declarative pipelines

1.1 Introduction and background

Instructor, Chris B. Behrens, has been a coder, technologist, and DevOps professional since before the term “DevOps” existed. He has worked with Jenkins since its inception. All code samples from the course are stored in a public GitHub repository:

https://github.com/FeynmanFan/declarative-pipelines

This repository is open for reading: it is possible to clone it, fork it, and use it freely. In a real business context, credentials would be necessary to clone a private repository — this point is mentioned but deliberately left aside in this introductory course.

The Jenkins instance used is running in a Docker container (version 2.479.2). Using Docker for Jenkins is entirely abstracted from the course experience, particularly because the pipelines are loaded from version control. The instance is launched with a local volume, which makes it easier to persist state outside the container. To learn more about this topic, the “Running Jenkins and Docker” course is recommended.

Note: Using Docker for Jenkins is touted as much superior to installing on bare metal. This is a highly recommended approach.

1.2 What “Declarative” means in Jenkins pipelines

Jenkins offers two syntaxes for pipelines:

1. Declarative Pipelines (Declarative Pipelines)
2. Scripted Pipelines (Scripted Pipelines)

Comparison of the two syntaxes

Declarative pipeline — Hello World:

pipeline {
    agent any
    stages {
        stage('Hello') {
            steps {
                echo 'Hello World'
            }
        }
    }
}

Scripted pipeline — Hello World:

node {
    stage('Hello') {
        echo 'Hello World'
    }
}

At first glance the two seem close, but this is misleading. The declarative pipeline has more structure. The difference is similar to that found in other areas of technology: a simple and accessible solution for the majority of use cases, versus a more complex solution for advanced users (power users).

This same pattern is found everywhere:

• High-level vs. low-level languages: C# is accessible and covers the majority of needs, but it has limits. For extreme cases, C++ or Assembly are required.
• Declarative vs. scripted pipelines: declarative pipelines cover the vast majority of DevOps CI/CD needs with a clear and structured syntax.

Real life example: full pipeline

Here is an example of a real pipeline showing both syntaxes, both performing the same steps:

1. Pull code from GitHub
2. Restoring dependencies
3. Static analysis
4. Build code
5. Running unit tests
6. Packaging for deployment
7. Sending results to a static analysis portal

The sample course is completely plugin-free, driven only by shell commands. This demonstrates the power of declarative pipelines in their purest form.

Conclusion on the choice

Declarative pipeline is the recommended modern standard. He is:

• More readable and self-documenting
• More restrictive (which limits errors)
• Best for team collaboration
• Natively supported by Blue Ocean and other Jenkins tools

The scripted pipeline is more flexible but requires more advanced Groovy expertise. It is preferred in very specific cases where the declarative is not enough.

1.3 Demo: Hello, Declarative Pipeline

Creating a first inline pipeline

In the Jenkins interface, we create a new build of type Pipeline (not Freestyle). We start with an inline script (online, directly in Jenkins) before moving it into a versioned file.

Here is the complete Hello World pipeline:

pipeline {
    agent any
    stages {
        stage('Hello') {
            steps {
                echo 'Hello World'
            }
        }
    }
}

Explanation of items:

• pipeline: the root element that contains the entire pipeline
• agent any: the pipeline can run on any available agent
• stages: container for all stages of the pipeline
• stage('Hello'): an internship with a descriptive label
• steps: container for the concrete actions of the course
• echo 'Hello World': a display statement in the build console

Added environment variables and the script block

We can enrich this pipeline with an environment block and a script block:

pipeline {
    agent any
    environment {
        SENTENCE = 'Four score and seven years ago'
    }
    stages {
        stage('Hello') {
            steps {
                echo 'Hello World'
                script {
                    def words = env.SENTENCE.split(' ')
                    for (word in words) {
                        echo word
                    }
                }
            }
        }
    }
}

Important points:

• The environment block declares a variable available in all stages of the pipeline
• The script block allows you to write arbitrary Groovy code inside a declarative stage
• The environment variable is accessible via env.VARIABLE_NAME
• If the variable was declared locally in the steps, its scope would be limited to this single step

The script block — an essential tool

The script block is a node that can reside inside a steps block. It contains Groovy code to automate more complex tasks. It is the bridge between the declarative world and the scripted world — we can do scripted inside the declarative.

1.4 Demo: Move the pipeline to a Jenkinsfile in GitHub

Why version the pipeline?

A pipeline that lives directly in Jenkins (inline) has a fundamental problem: there is a single global copy of the pipeline for all branches. If the code changes and the build must change to compile it correctly, we cannot have both states at the same time.

The solution is to place the Jenkinsfile alongside the code, in the Git repository, branch by branch. So:

• Branch feature/xyz can have a pipeline tailored to its needs
• The main branch keeps its pipeline stable
• Pipeline changes follow code changes in the same commit

Course scenario: Globomantics Inc.

In the educational scenario, you have been hired by Globomantics Inc. as a DevOps developer. The company has just acquired a software company and wants to formalize its informal build practices with Jenkins’ declarative syntax.

Setting up the Jenkinsfile in GitHub

1. Clone the GitHub repository locally:

   git clone https://github.com/FeynmanFan/declarative-pipelines.git
   cd declarative-pipelines
   

2. Create a pipelines folder to store the Jenkinsfiles:

   mkdir pipelines
   

3. Create the pipelines/simple-declarative.Jenkinsfile file with the contents of the pipeline

4. In Jenkins, modify the build configuration:

• Change source type from “Inline script” to “Pipeline script from SCM”
• Select Git
• Enter repository URL
• Specify Jenkinsfile name (ex: pipelines/simple-declarative.Jenkinsfile)

Contents of file simple-declarative.Jenkinsfile

pipeline {
    agent any
    triggers{
        cron('0 3 * * 1-5')
    }

    stages {
        stage('SCM'){
            steps{
                git branch: 'main', url: 'https://github.com/FeynmanFan/declarative-pipelines.git'
            }
        }
        stage('build') {
            steps {
                echo 'build the code'
                // build the code
            }
        }

        stage('Package') {
            when {
                expression {
                    return params.branch == "release"
                }
            }
            steps {
                echo 'Packaging the code for release'
            }
        }
    }
}

This file introduces several concepts: a cron trigger, a conditional stage with when, and the reference to a params.branch parameter.

1.5 Validation of declarative pipelines — the linter

Analogy with ReSharper

Before talking about the Jenkins linter, the trainer discusses ReSharper, a plugin for .NET developers that analyzes code in real time, identifies problems, offers suggestions for improvement and refactors automatically. ReSharper has played a key role in the adoption of LINQ: many developers learned LINQ thanks to automatic refactoring suggestions. This is an example of the impact a code quality tool can have.

Today, AI tools are taking over in this evolution, but the concept of a code analysis tool remains fundamental.

Linters

A linter is a tool that statically analyzes the text of a source file to identify problems and make suggestions. This is called static analysis.

For Jenkins pipelines:

• We can have a linter which analyzes the Jenkinsfile before executing it
• This helps detect syntax errors early in the development cycle
• This is a manual process (during development), not a build step

Why connect the linter to a specific Jenkins instance?

Unlike other linters that operate with their own rule base, the Jenkins linter must be connected to a specific Jenkins instance. The reason: What constitutes a valid pipeline depends on the configuration of the instance, specifically installed plugins.

Example: If the .NET SDK plugin is installed, .NET specific commands are valid in the pipeline. If this plugin is not installed, these same commands are invalid. The linter must therefore know the real state of the instance.

1.6 Demo: Using the linter on your pipeline

Installing the VS Code extension

In Visual Studio Code, install the Jenkins Pipeline Linter Connector extension.

This extension has several configuration parameters:

• Crumb URL: URL of your Jenkins instance to obtain the crumb (CSRF protection). Format: http://VOTRE_SERVEUR:PORT/crumbIssuer/api/json
• URL: URL of your Jenkins instance. Format: http://VOTRE_SERVEUR:PORT
• User: left empty if using an API token
• Password: the API token generated in Jenkins

Generate an API token in Jenkins

1. Connect to Jenkins
2. Click on the logged in user (top right)
3. Select Security
4. In the API Token section, click Add new Token
5. Name the token (ex: linter-token) and click Generate
6. Copy the token immediately — it will no longer be displayed

Using linter

Once configured, the linter can parse the Jenkinsfile opened in VS Code. If the syntax is correct, a confirmation message is displayed. In the event of an error, the problematic line and an explanatory message are provided.

1.7 Basic Syntax Summary

Here are all the elements covered in this first module:

The cron trigger

triggers {
    cron('0 3 * * 1-5')
}

This trigger allows you to schedule the execution of the pipeline. The cron syntax 0 3 * * 1-5 means: at 3:00 a.m., Monday to Friday.

Recommended resource for cron syntax: https://crontab.guru — the best tool for building cron expressions.

The three possible values for the trigger:

• cron: scheduled execution
• pollSCM: periodically checking the SCM for changes
• upstream: triggered by another pipeline

2. Build multi stage and multi branch pipelines

2.1 Multi-stage vs Multi-branch: clarification of terms

Multi-stage pipeline

From a technical point of view, all declarative pipelines are multi-stage in nature, even those that only implement a single stage. A multi-stage pipeline is simply a pipeline with several isolated stages that perform tasks in sequence to achieve a goal.

The characteristics that distinguish the internships from each other:

• Stage-scope variables: a variable declared in a stage is not accessible to other stages
• Different agents: each stage can run on a different agent
• Parallel execution: some stages can run in parallel
• Conditional execution: certain stages can be skipped depending on conditions

This last point is particularly important for merging similar pipelines. If two pipelines perform the same tasks with some differences, conditional stages allow them to be merged into one. However, spaghetti code must be avoided: do not merge pipelines that solve fundamentally different problems.

Multi-branch pipeline

A multi-branch pipeline is essentially an automation of a universal pattern in Jenkins. Without multi-branch, we generally pass the name of the branch as a pipeline parameter to build on several branches — a valid pattern and still used, but the formal “multi-branch” build type offers additional advantages.

How a multi-branch pipeline works: When creating a multi-branch build, instead of creating a single build, Jenkins scans the repository to find all branches that contain a Jenkinsfile. For each branch found, Jenkins automatically creates a separate build.

Advantage: instead of having to pass the branch name as a parameter and manually verify that the build works for each branch, Jenkins handles everything automatically. Anything that required manual intervention becomes a simple setup task.

2.2 Demo: Simple multi-stage pipeline with conditional stage

Scenario

Globomantics Inc. currently has separate builds for development and for preparing for production deployment. These builds share a lot of code in common. Management wants to eliminate the maintenance overhead of two separate builds.

The solution: create a conditional packaging stage which only executes on the release branch.

Added conditional stage with when

stage('Package') {
    when {
        expression {
            return params.branch == "release"
        }
    }
    steps {
        echo 'Packaging the code for release'
    }
}

Explanation:

• The when block must be placed above the steps block in the stage declaration
• expression allows evaluating any Groovy expression
• If the expression returns false, the stage is skipped (not executed)
• In console output, Jenkins displays a note that the stage was skipped

Added branch parameter

To make this stage truly functional, we add a branch parameter to the pipeline:

pipeline {
    agent any
    parameters {
        string(name: 'branch', defaultValue: 'main', description: 'The branch to fetch')
    }
    stages {
        stage('SCM') {
            steps {
                git branch: '${branch}', url: 'https://github.com/FeynmanFan/declarative-pipelines.git'
            }
        }
        stage('build') {
            steps {
                echo 'build the code'
            }
        }
        stage('Package') {
            when {
                expression {
                    return params.branch == "release"
                }
            }
            steps {
                echo 'Packaging the code for release'
            }
        }
    }
}

Thus:

• A build with branch=main will not run the Stage Package
• A build with branch=release will run the Stage Package

2.3 Parallelism in Jenkins pipelines

Maximize parallelism

Parallelism is one of the most valuable optimizations in CI/CD pipelines. The basic principle: maximize the simultaneous execution of courses as much as possible, within the limits imposed by dependencies.

Example of dependencies in a typical Globomantics pipeline

Consider a pipeline with the following steps (each takes 10 seconds):

1. checkout (SCM)
2. static analysis
3. build
4. unit tests
5. package

By analyzing the dependency graph:

• checkout is the starting point, everything depends on it
• static analysis only depends on checkout
• build depends on checkout
• unit tests depends on build
• package depends on: static analysis + build + unit tests (all must pass)

Parallelism opportunity: after checkout, we can execute static analysis and the sequence build → unit tests in parallel.

checkout
    ├── static analysis ─────────────────┐
    └── build → unit tests ──────────────┴── package

Without parallelism: 50 seconds (5 × 10s) With parallelism: 30 seconds (checkout 10s + max(static 10s, build+test 20s) + package 10s)

2.4 Demo: Simple parallelized build

Starting point

The trainer creates a copy of the existing pipeline and a new Jenkins build named “Hello Parallel”. To measure improvement, 3-second sleep instructions are added to each stage:

pipeline {
    agent any
    parameters {
        string(name: 'branch', defaultValue: 'main', description: 'The branch to fetch')
    }
    stages {
        stage('SCM') {
            steps {
                git branch: '${branch}', url: 'https://github.com/FeynmanFan/declarative-pipelines.git'
            }
        }
        stage('static analysis') {
            steps {
                echo 'perform static analysis'
                sleep time: 3, unit: 'SECONDS'
                echo 'sa complete'
            }
        }
        stage('build') {
            steps {
                echo 'build the code'
                sleep time: 3, unit: 'SECONDS'
                echo 'build complete'
            }
        }
        stage('unit test') {
            steps {
                echo 'execute unit tests'
                sleep time: 3, unit: 'SECONDS'
                echo 'unit tests complete'
            }
        }
        stage('Package') {
            steps {
                echo 'Packaging the code for release'
            }
        }
    }
}

Result without parallelism: ~43 seconds

Parallel pipeline transformation

The simple-parallel.Jenkinsfile file shows the parallelized structure:

pipeline {
    agent any
    parameters {
        string(name: 'branch',
               defaultValue: 'main',
               description: 'The branch to fetch for the pipeline')
    }

    stages {
        stage('SCM') {
            steps {
                git branch: '${branch}', url: 'https://github.com/FeynmanFan/declarative-pipelines.git'
            }
        }
        stage('parallel phases') {
            parallel {
                stage('static analysis') {
                    steps {
                        echo 'perform static analysis'
                        sleep time: 3, unit: 'SECONDS'
                        echo 'sa complete'
                    }
                }
                stage('build and test') {
                    stages {
                        stage('build') {
                            steps {
                                echo 'build the code'
                                sleep time: 3, unit: 'SECONDS'
                                echo 'build complete'
                            }
                        }
                        stage('unit test') {
                            steps {
                                echo 'execute unit tests'
                                sleep time: 3, unit: 'SECONDS'
                                echo 'unit tests complete'
                            }
                        }
                    }
                }
            }
        }
        stage('Package') {
            steps {
                echo 'Packaging the code for release'
            }
        }
    }
}

Key points of parallel syntax:

• A parent stage parallel phases wraps the two parallel paths
• The keyword parallel contains stages that run simultaneously
• To sequence stages within a parallel branch, we use a nested stages node
• Indentation is not required (unlike YAML), but is strongly recommended for readability

Result with parallelism: significant reduction in total execution time

2.5 Demo: Multi-branch pipeline

Creating a multi-branch build in Jenkins

1. In Jenkins, create a new build of type Multibranch Pipeline (not standard Pipeline)
2. Name the build (ex: “Hello Multi”)
3. In the configuration, enter the URL of the Git repository
4. Correct the name of Jenkinsfile if necessary
5. Save

Jenkins immediately launches a Scan Repository Log. It scans the repository to find all branches. In the course example, it finds the main and release branches, and automatically creates two builds:

• Hello Multi-hand
• Hello Multi-release

Problem with branch parameter in multi-branch context

In our pipeline, we had a branch parameter and an explicit SCM step:

parameters {
    string(name: 'branch', defaultValue: 'main', description: 'The branch to fetch')
}
stages {
    stage('SCM') {
        steps {
            git branch: '${branch}', url: '...'
        }
    }
}

In a multi-branch pipeline, Jenkins does not pass this parameter automatically. It interprets ${branch} literally, looking for a branch named ${branch}, which fails.

The solution: in a multi-branch pipeline, checkout is implicit. Jenkins automatically manages the checkout of the corresponding branch. You must:

1. Delete the parameters { string(name: 'branch', ...) } block
2. Delete explicit SCM stage
3. Let Jenkins handle the checkout implicitly

After this fix, the multi-branch pipeline works correctly, with each branch having its own automatic build.

2.6 The role of manual steps in DevOps pipelines

The maxim: partially automated is better than fully manual

This maxim, shared with customers, is fundamental: partially automated is much, much better than completely manual. Completely automating a long, established process can be painful, but even a pipeline full of manual steps is better than a completely manual process.

Why?

• A pipeline with manual steps documents at least what each step is supposed to do
• Completely manual processes tend to have steps forgotten or short-circuited under pressure
• Once the manual steps are in the pipeline, we can automate them one by one, gradually

Two types of manual steps

1. Manual by default (not yet automated): these are steps that remain manual simply because we have not yet taken the time to automate them. Today, the operator pushes the package to the deployment tool. Next week a script can do this automatically.

2. Manual for human reasons: certain steps require human intervention because they call on intuition, creativity, or human validation (e.g.: visually validate the result of a change, approve a new design before its promotion to production).

Code smell from persistent manual steps

When a manual step resists automation and continually requires human intervention, it is usually a smell indicating that the process is happening at the wrong time in the application lifecycle.

In Gitflow:

• We are working on feature branches
• We merge into a develop branch (higher degree of certainty)
• We then merge to main or release (even more certainty)

Builds typically trigger on a merge to a branch or on a branch, which represents a higher degree of certainty in code quality. Triggering validation processes at this time is more natural and reduces friction.

2.7 Demo: Pipeline with manual approval step

Worst case: a completely manual pipeline

The scenario: the first week at Globomantics, we create a pipeline representing the current process — entirely manual. Having the steps documented in a pipeline is already a big step forward.

File all-manual.Jenkinsfile:

pipeline {
    agent any

    stages {
        stage("static-analysis") {
            steps {
                echo 'static-analysis'

                script {
                    def number = input(
                        message: "Have you performed static analysis and that has resulted in no High or Critical issues?",
                        parameters: [
                            string(name: 'Value', description: 'Enter a value equally divisible by three.')
                        ]
                    )

                    def num = number.toInteger()

                    if (num % 3 == 0) {
                        echo "Divisible by three"
                    } else {
                        error "Not divisible by three"
                    }
                }
            }
        }

        stage("build") {
            steps {
                echo 'build'
            }
        }
        stage("unit test") {
            steps {
                echo 'unit test'
            }
        }
        stage("package") {
            steps {
                echo 'package'
            }
        }
    }
}

Operation:

• The input function interrupts the pipeline and waits for an operator response
• Operator sees a message in the Jenkins interface and must enter a value or click a button
• Pipeline resumes (or fails) based on response provided
• Here, the operator must enter a value divisible by 3 to validate that the static analysis was done correctly

Step input — basic syntax

input message: 'Avez-vous effectué l\'analyse statique ?', ok: 'Oui, continuer'

Or with additional parameters:

input(
    message: 'Approuvez-vous ce déploiement en production ?',
    ok: 'Oui, déployer',
    submitter: 'admin,ops-team',
    parameters: [
        choice(name: 'environment', choices: ['staging', 'production'])
    ]
)

3. Integrate declarative pipelines with build tools and notifications

3.1 Working with build tools in declarative pipelines

The Evolution of Jenkins Agent Provisioning

Before Docker, when we needed a new Jenkins agent, we had to:

• Install Jenkins on a new bare metal server, or
• Provision a new VM

These machines were persistent and required continuous maintenance: security patches, updates, and above all, the management of SDKs corresponding to the labels assigned to each agent.

The arrival of Docker changed everything:

• Jenkins can now provision a Docker agent dynamically
• Agent is created in real time, build runs on it, then agent is destroyed
• More compute consumption when it is not necessary
• Possibility of scale horizontally easily in case of queue
• End of manual agent management Jenkins

The best advantage of Docker for Jenkins: The agent configuration is entirely specified by the Docker image, itself defined by a Dockerfile. The agent specification and configuration are entirely hardwired — impossible to drift, impossible to forget to install an SDK, impossible to have agents that differ from each other.

The tools directive

The tools directive allows you to specify build tools (like Maven, JDK, Gradle) that will be auto-installed by Jenkins if necessary. This approach is relevant when you cannot use Docker for agents.

pipeline {
    agent any
    tools {
        maven 'maven-3.9.9'
    }
    stages {
        stage('Build') {
            steps {
                sh 'mvn --version'
            }
        }
    }
}

Important note: the tools directive requires that the tool be previously configured in Manage Jenkins → Tools. The name specified in the tools must exactly match the name configured in Jenkins.

3.2 Demo: Simple build with the tools directive (Maven)

Configuring Maven in Jenkins

1. Go to Manage Jenkins → Tools
2. Scroll down to the Maven section
3. Click Add Maven
4. Name the installation according to the version: maven-3.9.9
5. Leave auto-install checked
6. Click Save

Warning: even if a ”.NET SDK” section appears in the tools configuration, this does not mean that the tools directive supports .NET. Do not confuse the two.

File maven-build.Jenkinsfile

pipeline {
    agent any
    tools {
        maven 'maven-3.9.9'
    }
    stages {
        stage('Maven version') {
            steps {
                sh 'mvn --version'
            }
        }
    }
}

What happens at runtime: Jenkins hits declarative: tool installed directive in console logs. If Maven is not already installed on the agent, it automatically downloads and installs it before running the stages. Self-installation is only done once — subsequent builds reuse the existing installation.

3.3 Observability in pipelines

Analogy: legal discovery

In American legal proceedings and in most Western countries, there is a phase called discovery where all the facts that will be argued are discovered. A strategy sometimes employed is to drown the adversary in documents — potentially hiding a compromising detail in thousands of pages that a human being must go through.

Famous example: a lawsuit involving Valve Software, where the opposing company provided thousands of pages in Korean. A Korean intern translated them all and found a damning confession that resolved the lawsuit in Valve’s favor.

The lesson: information is only useful if it is easily observable. It’s the same principle with logs: having logs is good. But if you can’t find them easily or identify the important line in 100,000 entries, it’s not really useful.

Observability in the CI/CD context

The objective of builds (before packaging) is to create knowledge about the state of the code. If this knowledge is created but no one knows about it, it is not really knowledge.

For knowledge to be actionable, it must be highly observable. With the input step mentioned previously, if an operator’s approval is required but they are not informed of it, the process will not work.

Build notifications (like Slack) make build results immediately observable by the team.

3.4 Demo: Create a Slack notification for a build

Installing the Slack plugin

1. In Jenkins: Manage Jenkins → Plugins → Available
2. Search for slack
3. Install the Slack Notification plugin

Creating the Slack app

1. Go to https://api.slack.com/apps
2. Click Create an app → From a manifest
3. Select the target Slack workspace
4. On the Jenkins plugin page, copy the provided YAML manifest
5. Paste the manifest and create the app
6. Go to OAuth & Permissions → Install to Workspace
7. Authorize the requested permissions
8. Copy the Bot User OAuth Token (keep it in a safe place)

Configuring the plugin in Jenkins

1. Manage Jenkins → System (Ctrl+F “Slack”)
2. Workspace: enter the name of your Slack workspace
3. Credential: create a new credential of type Secret Text with the Bot Token
4. Test Connection to check
5. Save

File simple-slack.Jenkinsfile

pipeline {
    agent any
    stages {
        stage('Build') {
            steps {
                echo 'Build the code'
            }
        }
        stage('Notify') {
            steps {
                echo 'Send notification'
                slackSend channel: 'devops', message: 'Build has completed.'
            }
        }
    }
}

The slackSend step is provided by the Slack plugin. In particular, it accepts:

• channel: the target Slack channel (ex: #devops)
• message: the message to send
• color: the color of the notification (ex: 'good', 'warning', 'danger')

More complete example:

post {
    success {
        slackSend channel: '#devops',
                  color: 'good',
                  message: "Build ${currentBuild.fullDisplayName} réussi."
    }
    failure {
        slackSend channel: '#devops',
                  color: 'danger',
                  message: "Build ${currentBuild.fullDisplayName} ÉCHOUÉ."
    }
}

3.5 Demo: Manage artifacts in a build

Context

A Jenkins pipeline can perform purely probative actions (creating knowledge about the state of the code, such as unit tests) or produce a deployment artifact (a package ready to be deployed).

Potential issues with artifacts:

• The artifact is in the workspace, which may be ephemeral (Docker agents)
• Need a way to store and access the artifact after the build completes

The solution: the archiveArtifacts directive in the post block.

File simple-artifacts.Jenkinsfile

pipeline {
    agent any

    environment {
        ARTIFACT_SOURCE_DIRECTORY = "tests/*.xml"
    }
    stages {
        stage('Build') {
            steps {
                echo 'Build the code'
            }
        }
        stage('Test') {
            steps {
                echo 'Execute unit tests'
                sh 'mkdir -p tests && echo "test results" > tests/testresults.xml'
                error "Broken tests break the build"
            }
        }
    }
    post {
        always {
            archiveArtifacts artifacts: ARTIFACT_SOURCE_DIRECTORY, followSymlinks: false
        }
    }
}

Important points:

• The environment variable ARTIFACT_SOURCE_DIRECTORY defines the glob pattern of the files to be archived
• Block post { always { ... } } runs always, even if build fails
• This allows test results to be archived even when the build is in error
• The error "message" instruction intentionally forces the build to fail (to simulate broken tests)

Pattern glob for artifacts:

archiveArtifacts artifacts: '**/*.jar', followSymlinks: false  // tous les JARs
archiveArtifacts artifacts: 'target/*.war', followSymlinks: false  // un WAR spécifique
archiveArtifacts artifacts: 'reports/**/*', followSymlinks: false  // tous les rapports

After archiving, the artifact is accessible:

• Via the Jenkins interface (“Build Artifacts” tab of the build)
• Via Jenkins API
• By the following builds via copyArtifacts (Copy Artifact plugin)

3.6 The external resources model

Universal pattern for connecting between systems

There is a very common pattern for connecting two resources:

1. Go to resource B (target) and create the conditions for a trust relationship, in particular by creating an app
2. Resource B provides a secret (API key, token) for authentication
3. Go to resource A (source) and configure:

• The URL to connect to B
• The shared secret that A will present to B

This pattern is exactly what we followed with Slack, and it is the general way to connect resources on the internet.

Variant: in some cases (eg: Azure), the resource can simply trust the public key of the source resource. The source resource signs its request with its public key, which serves as a credential.

The constant is the credential. Finding what the credential is in any login system helps to understand and follow the process.

3.7 Demo: Trigger an Azure DevOps pipeline from Jenkins

Globomantics Background

Globomantics has a separate division for deployment operations. These teams use Azure DevOps rather than Jenkins. The connection between Jenkins and Azure DevOps is currently manual — it needs to be automated.

Required credential: PAT (Personal Access Token)

An Azure DevOps PAT (Personal Access Token) is the equivalent of a programmatic password for the Azure DevOps API.

IMPORTANT: Secrets should NEVER be in version control.

The PAT must be:

1. Created in Azure DevOps (User Settings → Personal Access Tokens)
2. Stored in Jenkins as credential of type Secret Text (with ID ado-pat)
3. Referenced in the pipeline via credentials('ado-pat')

File simple-trigger-external.Jenkinsfile

pipeline {
    agent any

    environment {
        ARTIFACT_SOURCE_DIRECTORY = "tests/*.xml"
        ORG = 'OurWebCompany'
        PROJECT = 'OurNewApplication'
        PIPELINE_ID = '50'
        PAT = credentials('ado-pat')
    }
    stages {
        stage('Build') {
            steps {
                echo 'Build the code'
            }
        }
        stage('Test') {
            steps {
                echo 'Execute unit tests'
                sh 'mkdir -p tests && echo "test results" > tests/testresults.xml'
                // error "Broken tests break the build"
            }
        }
    }
    post {
        always {
            archiveArtifacts artifacts: ARTIFACT_SOURCE_DIRECTORY, followSymlinks: false

            echo 'Upload the artifact to Azure Storage'

            script {
                // POST https://dev.azure.com/{organization}/{project}/_apis/build/builds?api-version=7.1
                def targetURL = "https://dev.azure.com/${env.ORG}/${env.PROJECT}/_apis/build/builds?definitionId=${env.PIPELINE_ID}&api-version=7.1"

                def authValue = ":${env.PAT}".bytes.encodeBase64().toString()

                sh "curl -X POST --header 'Content-Type: application/json' --header 'Content-Length:0' --header 'Authorization: Basic $authValue' --url '$targetURL'"
            }
        }
    }
}

Azure DevOps trigger explanation:

• Azure DevOps REST API allows triggering a build via an HTTP POST request
• Target URL is dynamically constructed with environment variables
• Authentication is done with the PAT encoded in Base64 in an Authorization: Basic header
• Base64 encoding ":${env.PAT}" (with a : before the PAT) follows the Basic Auth convention where the format is username:password — here the username is empty

4. Working with environment variables and secrets

4.1 Secrets have no place in version control

The history of version control

The trainer, having worked before version control existed, recalls the development:

• Without version control: backup folders created “when you think about it”, non-existent diff operations, nothing like blame
• Visual SourceSafe: the first VCS used — its database corrupted at the slightest problem, probably putting Microsoft developers off adoption of version control for years
• Subversion: less bad
• Git (2005): created by Linus Torvalds, the best VCS ever designed

Git is specifically designed to be:

• Hard to lose a commit: removing a commit is a lot of effort, especially if it’s not the last one
• Widely available and accessible: add a new remote, push, and go to lunch

Why secrets should NEVER be in Git

Every Git capability is in direct contradiction to the goal of removing something from the repository. This is precisely what you should do when you have accidentally committed a secret.

Even if you remove the secret and commit, the problems persist:

• Secret exists in Git history
• If the repository has been cloned (even once), the secret exists elsewhere
• Git is designed to be pushed to multiple remotes
• Anyone with access to the repository (or a fork) can see the history

Absolute rule: literal secret values ​​must never appear in a Jenkinsfile or any other versioned file. Always use references to Jenkins credentials (credentials('credential-name')).

4.2 Demo: Working with secrets in Jenkins

Credential types in Jenkins

In Manage Jenkins → Credentials → Global, you can create different types of credentials:

Scoping of credentials with Domains

In Jenkins, you can create credential domains to restrict access according to the URI. Example: a domain with a rule on github.com will only provide credentials for connections to GitHub.

In the demo, a custom domain can have rules based on:

• The URI scheme (http, https, ssh…)
• The hostname
• The port
• The path

Reference a credential in the pipeline

Simple method (via the environment directive):

environment {
    PAT = credentials('ado-pat')
}

Method with lifetime check (withCredentials):

withCredentials([string(credentialsId: 'ado-pat', variable: 'PAT')]) {
    sh "curl -H 'Authorization: Bearer ${PAT}' https://api.exemple.com"
}

The key difference: withCredentials allows you to precisely control the length of time the secret is available. This is an important security principle — a secret should only be available for as short a time as possible.

4.3 Demo: Using environment variables in pipelines

Environment variables automatically available

Jenkins provides many environment variables to your pipeline, without additional configuration.

all-evs.Jenkinsfile file — show all environment variables:

pipeline {
    agent any
    stages {
        stage('List Environment Variables') {
            steps {
                script {
                    echo "Listing all Jenkins built-in environment variables:"
                }
                sh '''
                echo "Environment Variables:"
                echo "-----------------------"
                # Iterate through all environment variables
                printenv | sort
                '''
            }
        }
    }
}

Main Jenkins environment variables:

Complete documentation: http://VOTRE_INSTANCE:PORT/pipeline-syntax/globals#env

Trainer tip: take 5 minutes to go through all of these variables. Create a memory hook for those you might need. Most are pretty obscure, but you never know when you’ll need a difficult-to-trace environment element.

Example of practical use:

post {
    success {
        sh "git tag deploy-${env.BUILD_NUMBER} ${env.GIT_COMMIT}"
        sh "git push origin deploy-${env.BUILD_NUMBER}"
    }
}

4.4 Configure your pipeline

Natively supported parameter types

Jenkins supports the following parameter types:

Warning on the password parameter

The password type hides the value in the logs, BUT:

• If you use echo ${params.MY_PASSWORD}, the value will appear in plain text in the log
• If a plugin displays the value in debug mode, it will appear
• Masking is applied after the fact for most pipeline items

Strong recommendation from the trainer: do not use the password type. Instead use Jenkins credentials (credentials('credential-name')) which are designed for security.

4.5 Demo: Pipeline with build parameters

File simple-parameter-types.Jenkinsfile

pipeline {
    agent any

    parameters {
        text(name: 'multiline',
             defaultValue: 'This is a default multiline value for the parameter')

        booleanParam(name: 'isAdmin',
                     defaultValue: false,
                     description: 'Whether or not the current user is admin')

        choice(name: 'targetEnvironment',
               choices: ['dev', 'staging', 'qa', 'production'],
               description: "The target environment")
    }

    stages {
        stage('pre-flight') {
            steps {
                echo "Text: ${params.multiline}"
                echo "Is current user admin? ${params.isAdmin ? 'Yes' : 'No'}"
                echo "Target environment: ${params.targetEnvironment}"
            }
        }
    }
}

Auto-generated user interface: When a pipeline has parameters, the Jenkins interface replaces the “Build Now” button with “Build with Parameters”. By clicking, the user sees:

• A wide text field for the text parameter (multiline)
• A checkbox for the booleanParam parameter
• A dropdown for the choice parameter

The Groovy ternary operator:

echo "Is current user admin? ${params.isAdmin ? 'Yes' : 'No'}"

This syntax — condition? value_if_true : value_if_false — is the ternary operator. This is a trivial use here, but we can use this pattern for flow control, like in a when condition:

when {
    expression {
        return params.targetEnvironment == 'production' && params.isAdmin
    }
}

5. Fix errors and improve pipeline resilience

5.1 Error handling in declarative pipelines

The try-catch-finally structure

If you know a programming language with structured error handling, the pattern is familiar:

try {
    // Code risqué — peut lever une exception
    sh 'mvn clean install'
} catch (Exception e) {
    // Exécuté UNIQUEMENT si une exception est levée
    echo "Erreur : ${e.message}"
    // Nettoyage spécifique à l'état d'erreur
} finally {
    // Exécuté TOUJOURS, qu'il y ait eu erreur ou non
    // Nettoyage général, fermeture de connexions, etc.
}

Matching declarative post blocks

Available post block types

post {
    always {
        // S'exécute toujours, quoi qu'il arrive
    }
    success {
        // S'exécute uniquement si le pipeline a réussi
    }
    failure {
        // S'exécute uniquement si le pipeline a échoué
    }
    unstable {
        // S'exécute si le build est marqué "unstable" (ex: tests failures)
    }
    changed {
        // S'exécute si le statut du build a changé par rapport au précédent
    }
    fixed {
        // S'exécute si le build précédent avait échoué et celui-ci réussit
    }
    regression {
        // S'exécute si le build précédent avait réussi et celui-ci échoue
    }
    aborted {
        // S'exécute si le build a été annulé manuellement
    }
    cleanup {
        // S'exécute toujours, après tous les autres blocs post
    }
}

Limitation compared to try-catch

A notable limitation: with post blocks, there is only one failure block for the entire pipeline. In traditional try-catch, we can have different catch blocks for different exceptions. In the failure block, we can inspect currentBuild.result or analyze currentBuild.rawBuild.getCause() to distinguish the causes of failure.

5.2 Demo: Security cleanup with error handling

Scenario

A pipeline that decrypts a file, performs an ETL process, and should handle security errors properly. The critical point: If decryption fails halfway, there may be a clear text file on the hard drive — a major security risk.

File simple-errorhandling.Jenkinsfile

pipeline {
    agent any

    stages {
        stage('Decrypt') {
            steps {
                script {
                    try {
                        echo "Starting decryption process..."

                        // Simulate decryption operation
                        echo "Decrypting file 'encrypted_file.dat'..."

                        // Simulate an error (uncomment to test error handling)
                        throw new Exception("Decryption failed partway through, and now there is a plain-text file sitting on the hard drive")

                        echo "Decryption successful! File saved as 'decrypted_file.dat'"
                    } catch (Exception e) {
                        echo "Error during decryption: ${e.message}"
                        echo "Delete the files because we don't know what state they're in"

                        throw e // Rethrow the exception to propagate the error
                    } finally {
                        echo "Delete the copy of the encrypted file, but leave the plaintext file in place for the next stage"
                    }
                }
            }
        }

        stage('ETL') {
            steps {
                script {
                    try {
                        echo "Starting ETL process..."

                        echo "Checking whether previous state succeeded and plaintext file exists"

                        // Simulate data load into the database
                        echo "Loading 'decrypted_file.dat' into the database..."

                        // Simulate an error (uncomment to test error handling):
                        // throw new Exception("Database connection timeout")

                        echo "ETL process completed successfully!"
                    } catch (Exception e) {
                        echo "Error during ETL: ${e.message}"
                        throw e // Rethrow the exception to propagate the error
                    } finally {
                        echo "Delete the entire workspace"
                    }
                }
            }
        }
    }

    post {
        failure {
            echo "Pipeline failed. Executing failure handling steps..."
            echo "Sending alert to administrators..."
        }
    }
}

Execution flow analysis:

Case: Decryption fails

1. throw new Exception(...) is thrown in the try block of the Decrypt stage
2. The code after the throw (success message) is skipped
3. The catch block executes: error log + deletion of the clear text file (mitigation of the security risk)
4. The exception is thrown (throw e) — the pipeline continues to propagate the error
5. finally block executes: deleting encrypted file (now useless)
6. The ETL stage is skipped (because of the exception propagated)
7. Block post { failure { ... } } executes: notification to administrators

Case: Decryption succeeds and ETL fails

1. The Decrypt course ends normally
2. finally block deletes encrypted file
3. The ETL stage starts
4. If an error occurs in ETL: catch logs the error, finally deletes the entire workspace
5. Block post { failure { ... } } executes

Why doesn’t finally delete the plain text file if everything is fine?

The finally block of the Decrypt stage only deletes the encrypted file. The plain text file is left in place for the ETL stage to process. If we wanted to hold the workspace for investigation in the event of a failure, we could put the cleanup in the post { success { ... } } block rather than in always or finally.

5.3 You don’t need Retry — but here’s how to do it

The soapbox on Retry

The trainer has a strong opinion on retry: it is generally a code smell which indicates a problem elsewhere.

Analogy: when an elevator does not arrive, we press the button again. For what ? Because the elevator is a black box — we have no visibility into what is happening inside. The opposite should be true with CI/CD builds.

The typical situation when using Retry: a resource is unavailable. Classic example: a junior developer writes retry code to connect to a database. The real question to ask is not “how do I try again?” but “why is the database unavailable?” The point of a database, versus a flat file on a network share, is precisely to create reliable availability. If not, that’s the problem at hand.

The legitimate exception: when you access an external resource that you do not control (a third-party API). These services may be temporarily overloaded. In this case, trying again is sometimes the best option.

Timeout

timeout is more universally useful. It prevents a build from being stuck indefinitely waiting for a response that may never come.

5.4 Demo: Retry and Timeout

File simple-retry-timeout.Jenkinsfile

pipeline {
    agent any

    environment {
        RETRY_COUNT = '0' // Initialize retry count to 0
    }

    stages {
        stage('Simulate External Connection') {
            steps {
                script {
                    retry(3) { // Retry up to 3 times if this block fails
                        // Increment the RETRY_COUNT environment variable
                        env.RETRY_COUNT = (env.RETRY_COUNT.toInteger() + 1).toString()

                        timeout(time: 90, unit: 'SECONDS') { // Timeout each attempt after 90 seconds
                            echo "Attempt #${env.RETRY_COUNT}: Connecting to external resource..."

                            // Simulated command: Replace with real connection command
                            sh '''
                                echo "Trying to connect..."
                                sleep 100  # Simulate a long-running connection (100 seconds)
                                echo "Connection failed!"
                                exit 1    # Simulate failure for retry
                            '''
                        }
                    }
                }
            }
        }
    }

    post {
        success {
            echo "Connection successful after ${env.RETRY_COUNT} attempt(s)!"
        }
        failure {
            echo "Connection failed after ${env.RETRY_COUNT} attempt(s)."
        }
    }
}

Pipeline analysis:

• RETRY_COUNT: environment variable to track the number of attempts (useful for diagnostic logs)
• retry(3): retry up to 3 times if the block fails
• timeout(time: 90, unit: 'SECONDS'): each attempt has 90 seconds to complete
• sleep 100 causes the timeout to exceed 90 seconds, triggering a retry

Nesting order is important:

// Option 1 : chaque retry a 90 secondes
retry(3) {
    timeout(time: 90, unit: 'SECONDS') {
        // ...
    }
}

// Option 2 : tout le bloc retry a 90 secondes au total
timeout(time: 90, unit: 'SECONDS') {
    retry(3) {
        // ...
    }
}

Syntax for retry in a script block (without the declarative retry):

script {
    def success = false
    def retryCount = 0
    def maxRetries = 3

    while (!success && retryCount < maxRetries) {
        try {
            retryCount++
            // Tentative de connexion
            sh 'curl https://api.exemple.com/data'
            success = true
        } catch (Exception e) {
            echo "Tentative ${retryCount} échouée : ${e.message}"
            if (retryCount >= maxRetries) {
                throw e
            }
            sleep(time: 5, unit: 'SECONDS')
        }
    }
}

Example of timeout for a deployment:

timeout(time: 10, unit: 'MINUTES') {
    sh './deploy.sh'
}

Other values ​​for unit: 'HOURS', 'MINUTES', 'SECONDS', 'NANOSECONDS', 'MICROSECONDS', 'MILLISECONDS'.

5.5 Course Summary

This course covered a wide range of concepts about Jenkins declarative pipelines:

Module 1 — The basics:

• The difference between declarative and scripted pipelines
• The fundamental structure: pipeline, agent, stages, stage, steps, script, environment, triggers
• Creating and validating a pipeline with the VS Code linter
• Moving the pipeline to a versioned Jenkinsfile in GitHub

Module 2 — Multi-stage and multi-branch:

• Conditional stages with when
• Parallelism with parallel
• Automatic multi-branch pipelines
• Manual steps with input

Module 3 — Integration with tools and notifications:

• The tools directive for build tools (Maven)
• Slack notifications
• Archiving artifacts with archiveArtifacts
• Integration with Azure DevOps via REST API

Module 4 — Environment variables and secrets:

• Jenkins credentials and their importance for security
• Jenkins automatic environment variables
• Pipeline parameter types

Module 5 — Resilience and error handling:

• The post blocks (always, success, failure, etc.)
• Error handling with try-catch-finally in the script block
• The use of retry and timeout

Final Trainer Recommendation:

Create a personal library of practice builds, drawing on all the examples in the Git repository: https://github.com/FeynmanFan/declarative-pipelines

6. Quick reference — Declarative pipeline syntax

Complete structure of a declarative pipeline

pipeline {
    // OBLIGATOIRE : spécification de l'agent
    agent any
    // ou : agent { label 'maven-agent' }
    // ou : agent none (si chaque stage spécifie son propre agent)

    // OPTIONNEL : déclencheurs automatiques
    triggers {
        cron('0 3 * * 1-5')
        pollSCM('H/15 * * * *')
    }

    // OPTIONNEL : paramètres de build
    parameters {
        string(name: 'BRANCH', defaultValue: 'main', description: 'Branche à builder')
        booleanParam(name: 'DEPLOY', defaultValue: false, description: 'Déployer ?')
        choice(name: 'ENV', choices: ['dev', 'staging', 'prod'], description: 'Environnement')
    }

    // OPTIONNEL : variables d'environnement globales
    environment {
        APP_NAME = 'monapp'
        VERSION = '1.0.0'
        API_KEY = credentials('mon-api-key')
    }

    // OPTIONNEL : outils de build
    tools {
        maven 'maven-3.9.9'
        jdk 'jdk-17'
    }

    // OBLIGATOIRE : les stages
    stages {
        stage('Checkout') {
            steps {
                git branch: 'main', url: 'https://github.com/org/repo.git'
            }
        }

        stage('Build') {
            environment {
                // Variable locale à ce stage seulement
                BUILD_OPTS = '--no-tests'
            }
            steps {
                sh 'mvn clean package ${BUILD_OPTS}'
            }
        }

        stage('Parallel Quality') {
            parallel {
                stage('Static Analysis') {
                    steps {
                        sh 'mvn checkstyle:check'
                    }
                }
                stage('Unit Tests') {
                    steps {
                        sh 'mvn test'
                    }
                }
            }
        }

        stage('Deploy to Staging') {
            when {
                expression {
                    return params.DEPLOY && params.ENV == 'staging'
                }
            }
            steps {
                echo 'Deploying to staging...'
            }
        }

        stage('Manual Approval') {
            steps {
                input message: 'Approuver le déploiement en production ?',
                      ok: 'Oui, déployer'
            }
        }
    }

    // OPTIONNEL : actions post-build
    post {
        always {
            archiveArtifacts artifacts: 'target/*.jar', followSymlinks: false
            junit 'target/surefire-reports/*.xml'
        }
        success {
            slackSend channel: '#devops', color: 'good', message: "Build ${BUILD_NUMBER} réussi"
        }
        failure {
            slackSend channel: '#devops', color: 'danger', message: "Build ${BUILD_NUMBER} ÉCHOUÉ"
            emailext to: 'equipe@exemple.com',
                     subject: "Build Jenkins échoué : ${JOB_NAME}",
                     body: "Le build ${BUILD_URL} a échoué."
        }
        cleanup {
            cleanWs() // Nettoyer le workspace
        }
    }
}

Useful links and resources

Search Terms

declarative · jenkins · pipelines · ci/cd · git · devops · pipeline · manual · multi-branch · environment · linter · parameter · secrets · tools · types · variables · added · block · branch · context · credential · globomantics · multi-stage · retry