Building a Modern CI CD Pipeline with Jenkins

Table of Contents

1. Course Overview
2. Start with Jenkins Scripted Pipelines

• 2.1 Introduction and learning path
• 2.2 Course pipeline and GitHub repositories
• 2.3 Configure Jenkins — Key concepts
• 2.4 Demo: Connect Jenkins to GitHub
• 2.5 Scripted Pipelines Hello World
• 2.6 Demo: Run a Scripted Pipeline
• 2.7 Module 2 Summary

3. Building and testing code

• 3.1 Introduction
• 3.2 Demo: Build and push a Docker container
• 3.3 Unit testing workflow and Multibranch Pipelines
• 3.4 Demo: Run unit tests
• 3.5 Module 3 Summary

4. Integrate container security and compliance

• 4.1 Introduction
• 4.2 Demo: Push a Docker container to a registry
• 4.3 Clear Container Scanner
• 4.4 Demo: Clear
• 4.5 Grype Container Scanner (Anchore)
• 4.6 Demo: Grype and stages in parallel
• 4.7 Module 4 Summary

5. Implement continuous deployment pipelines

• 5.1 Introduction
• 5.2 Conditions when and input steps
• 5.3 Demo: Conditions when and input steps
• 5.4 Module 5 Summary

6. Troubleshooting and Improving Jenkins Pipelines

• 6.1 Introduction
• 6.2 Demo: The Declarative Directive Generator
• 6.3 Jenkins Shared Libraries
• 6.4 Demo: Basic Groovy functions in a shared library
• 6.5 Demo: Advanced Groovy Features
• 6.6 Demo: Complete pipelines in a shared library
• 6.7 Module 6 Summary

1. Course Overview

This course, led by Christopher Blackden, is aimed at DevOps engineers or software developers who want to write pipelines capable of building, testing, and deploying their containerized applications to a Kubernetes cluster.

What is covered

• The syntax and functionality of Jenkins Pipeline code
• Using open-source container scanning tools (Grype, Clear)
• Creating reusable Shared Libraries for Jenkins Pipelines
• Using flow control steps and conditions to control deployments

Prerequisites

• Be familiar with basic Jenkins concepts
• Be comfortable using the command line

General course pipeline

The course pipeline follows this overall flow:

GitHub (code source)
    ↓
Pull des changements
    ↓
Construction des conteneurs Docker
    ↓
Tests unitaires (pytest)
    ↓
Scanning de sécurité (Clair + Grype en parallèle)
    ↓
Déploiement vers Kubernetes (QA → PROD avec approbation)

2. Getting started with Jenkins Scripted Pipelines

2.1 Introduction and learning path

This module introduces Jenkins Scripted Pipelines. There is a whole Jenkins course at Pluralsight, and this course is a continuation of:

• Getting Started with Jenkins — introduction to the basics
• Using Declarative Jenkins Pipelines — declarative pipelines
• Automating Jenkins with Groovy — automation with Groovy

If you have already gone through these courses or if these topics do not apply to you, you are in the right place.

2.2 Course pipeline and GitHub repositories

Application used: azure-voting-app-redis

The course uses an application forked from Azure Samples: azure-voting-app-redis. This application was chosen for several reasons:

• It is relatively simple
• All deployment is in one file
• It has a docker-compose.yaml to easily build the containers
• It is designed to be deployed to AKS (Azure Kubernetes Service)

Important: In a DevOps approach, we do not want to make application modifications at the same time as we add deployment instructions. We assume that the development team has created a working application, and our role is only to deploy it in a stable and secure manner.

There is also a jenkins-tutorial folder in this repository with some shell scripts that install Jenkins from a Docker file — these files are ignored in the course, as we will build our own deployment instructions in a Jenkinsfile.

Shared repository: demo-shared-pipeline

A second repository will be used later in the course. It contains shared deployment instructions (Shared Library). It will be presented in detail in module 6.

2.3 Configure Jenkins — Key Concepts

Jenkins: Flexibility and extensibility

Jenkins is an extremely flexible tool:

• Works on Windows, Linux, macOS
• Can run containerized via Kubernetes or Elastic Container Service
• Can run locally
• Is open-source with a large number of plugins

Freestyle Jobs vs. Scripted Pipelines

Scripted Pipelines are configured via code using a Groovy DSL. They are :

• Much more readable
• More maintainable and scalable
• Easy to share
• Versionable in source control in the same way as application code
• Easy to recover in case of disaster (the pipeline is already in the source code)

Basic definitions

• Pipeline: A series of tasks necessary to build, test and deploy an application
• Agent: Defines which Jenkins build agent should run the pipeline. Controls the execution environment
• Stage: A section of the pipeline — a single unit of work (e.g.: build, test, deploy)
• Step: Specific instructions within a stage. Ex.: sh 'make' and sh 'make install' are two distinct steps in a “Build” stage

Jenkins Reference Documentation

• Jenkins Pipeline: High-level pipeline documentation
• Pipeline Syntax: Reference on building a pipeline
• Pipeline Steps Reference: Reference of all steps available by plugin

The Pipeline Steps Reference is particularly useful — it contains hundreds of steps available via Jenkins plugins (e.g.: Slack notifications, various integrations). If a tool is not covered in the course, it is often because there already exists a Jenkins plugin to integrate it.

Location of Jenkinsfile in a repository

The best practice is to place the Jenkinsfile at the root of the repository, at the same level as the Dockerfile, the README, and the configuration files:

mon-application/
├── src/
├── Dockerfile
├── docker-compose.yaml
├── README.md
└── Jenkinsfile          ← ici

It is possible to change the name of the file or place it in a subdirectory by configuring the Script Path in the Jenkins job configuration.

2.4 Demo: Connect Jenkins to GitHub

Step 1 — Create a Personal Access Token GitHub

1. In Jenkins: Manage Jenkins → System → section GitHub
2. Click the ? (question mark) icon next to Credentials to see what Jenkins expects
3. Jenkins requires a Personal Access Token with permissions:

• admin:repo_hook
• repo
• repo:status

4. In GitHub: generate a new token with these permissions
5. Give the token an expiration date (good practice: regular key rotation — e.g. 60 days)
6. Copy the token immediately (visible only once)

Security best practice: Always use an expiration date for tokens. Never choose “No expiration”.

Step 2 — Add the token to Jenkins

1. In Jenkins, click Add → Jenkins
2. Choose type Secret text
3. Paste the token
4. Give a descriptive ID, e.g. : github-pat (PAT = Personal Access Token)
5. Click on Test connection to verify that the credentials are valid

Step 3 — The Pipeline plugin

The Pipeline plugin is already included with Jenkins by default. There is no need to install it separately. However, there may be updates available, which you are advised to apply regularly.

2.5 Scripted Pipelines Hello World

Basic structure of a Jenkinsfile

A Jenkinsfile (or Jenkins Pipeline document) is placed at the root of the repository. It contains a pipeline block with several sub-blocks:

pipeline {
    agent any
    stages {
        stage('Hello') {
            steps {
                echo 'Hello World'
            }
        }
    }
}

Explanations:

• pipeline { }: Mandatory root block
• agent any: The agent is required. any means the pipeline can run on any available agent. You can specify more precise agents (label, Docker, etc.)
• stages { }: Contains all stages in the pipeline
• stage('Hello') { }: A stage named “Hello”
• steps { }: Contains the stage instructions
• echo 'Hello World': A Jenkins Pipeline step — different from a shell script

Important point: echo 'Hello World' is a Jenkins Pipeline step, not a shell script. To run a shell script, you must use the sh step.

Difference between echo and sh

// Étape Jenkins (echo intégré)
echo 'Hello World'

// Étape shell (bash)
sh script: 'echo Hello World'

// Étape shell avec script multi-lignes
sh script: """
    echo Hello World
    docker compose build
"""

The distinction is important and can be confusing. The Jenkins Pipeline Steps Reference documentation clarifies which steps are available and how to use them.

2.6 Demo: Running a Scripted Pipeline

Create a new Pipeline job in Jenkins

1. Manage Jenkins → New Item
2. Name the job (e.g.: Hello World)
3. Choose type Pipeline → OK

Configuration options

• Build Triggers: Trigger the build on other jobs, GitHub polling, or via a REST API
• Pipeline script: Script entered directly into the interface
• Pipeline script from SCM: The script is read from the Git repository (recommended in production)

Hello World example executed

pipeline {
    agent any
    stages {
        stage('Hello') {
            steps {
                echo 'Hello World'
            }
        }
    }
}

Execution shows in logs: Hello World

Extended pipeline with 3 stages

pipeline {
    agent any
    stages {
        stage('Hello') {
            steps {
                echo 'Hello World'
            }
        }
        stage('Goodbye') {
            steps {
                echo 'Goodbye World'
            }
        }
        stage('Ok') {
            steps {
                sleep 5
                echo 'I\'m ok'
            }
        }
    }
}

Important points:

• Each stage appears in the Jenkins interface with its own logs
• Console Output aggregates all logs in execution order
• Step sleep 5 pauses the pipeline for 5 seconds
• The escape character \' allows an apostrophe to be included in the string

Navigating the Jenkins interface

• Click on the stages in the graphic view to directly access the stage logs
• Click on the green/red icon next to a build to access the Console Output directly

2.7 Module 2 Summary

This module covered:

• Introduction to Scripted Pipelines and their advantages vs. Freestyle Jobs
• Creating pipeline code and the anatomy of a Jenkinsfile
• Creating a basic Jenkinsfile with the pipeline syntax generator
• Configuring GitHub integration (Personal Access Token, Jenkins credentials)
• Jenkins documentation — where to look for answers in case of a crash

3. Building and testing the code

3.1 Introduction

This module covers building and testing code:

1. Build the container by running shell scripts in pipeline scripts
2. Run unit tests against this container
3. Learn Multibranch Pipelines to manage multiple branches simultaneously
4. Learn the post blocks (success and failure) to control the behavior after each stage

In the progress of the course, we are at the “building the Docker container” step — we pull the application from GitHub and build the containers.

3.2 Demo: Build and push a Docker container

The initial Jenkinsfile

On the main branch of the azure-voting-app-redis application, a Jenkinsfile was added with two stages:

pipeline {
    agent any
    stages {
        stage('Verify Branch') {
            steps {
                echo "$GIT_BRANCH"
            }
        }
        stage('Docker Build') {
            steps {
                sh script: """
                    docker compose build
                """
            }
        }
    }
}

Explanations:

• "$GIT_BRANCH": Built-in Jenkins environment variable that displays the current branch
• sh script: """ ... """: Runs a bash script. Triple quotes allow multi-line scripts
• docker compose build: Builds all containers defined in docker-compose.yaml

Configure a Pipeline from SCM (Source Code Management)

1. New Item → name azure-voting-app-redis → type Pipeline → OK
2. Go down to the Pipeline section
3. Change from Pipeline script to Pipeline script from SCM
4. Choose Git as SCM
5. Enter the HTTPS URL of the repository
6. Select previously configured GitHub credentials
7. Change the Branch Specifier to match the main branch
8. Check that the Script Path is Jenkinsfile (default value)

Note: The Script Path allows you to specify a different location or file name for the Jenkinsfile. By default, Jenkins looks for a file named Jenkinsfile at the root of the repository.

Running the build

After Build Now, Jenkins performs:

1. SCM Checkout — Retrieving code from GitHub
2. Verify Branch — Displays the branch name (main)
3. Docker Build — Runs docker compose build (building images)

If the images were pre-built, Docker will use the cache and the build will be very fast.

3.3 Unit testing workflow and Multibranch Pipelines

The problem: multiple developers, multiple branches

In a team project, several developers work simultaneously on the same repository, each on their own branch. How to ensure that:

• Can each developer run their tests without impacting others?
• The tests are isolated by branch?

Solution: Multibranch Pipeline

A Multibranch Pipeline treats each branch as its own Jenkins project and runs the same Jenkinsfile on each. Benefits :

• Tests run on each branch independently
• New testing features can be added to one branch without impacting others
• All changes are isolated to their branch

Unit testing workflow (concept)

Application (nouvelles fonctionnalités)
    ↓
Tests unitaires
    ├── SUCCÈS → Continuer le déploiement / push vers le registry
    └── ÉCHEC → Arrêter le pipeline (ne pas introduire de bugs)

3.4 Demo: Running Unit Tests

Jenkinsfile with unit tests and post blocks

On the feature/add-tests branch, the Jenkinsfile is enriched with new courses:

pipeline {
    agent any
    stages {
        stage('Verify Branch') {
            steps {
                echo "$GIT_BRANCH"
            }
        }
        stage('Docker Build') {
            steps {
                sh script: """
                    docker compose build
                """
            }
        }
        stage('Start App') {
            steps {
                sh script: """
                    docker compose up -d
                """
            }
        }
        stage('Run Tests') {
            steps {
                sh script: """
                    pytest tests/
                """
            }
            post {
                success {
                    echo 'Tests passed!'
                }
                failure {
                    echo 'Tests failed'
                }
            }
        }
    }
    post {
        always {
            sh script: """
                docker compose down
            """
        }
    }
}

Key points:

• docker compose up -d: Starts the application in the background (-d = detached mode)
• pytest tests/: Runs Python tests in the tests/ directory
• Post block in a stage: The post block inside a stage is scoped to this stage only
• success: Executed if the internship succeeds
• failure: Executed if the stage fails
• Global post block: The post block outside the stages block is scoped to the entire pipeline
• always: Executed no matter what (success, failure, etc.). Ideal for cleaning operations

Example of a pytest test that intentionally fails

# tests/test_sample.py
def test_math():
    assert 3 + 1 == 5  # Échec intentionnel : 4 ≠ 5

The error produced: AssertionError: assert 4 == 5

The pipeline then displays Tests failed (via the failure post block), but the post { always { ... } } block still executes and stops the Docker Compose stack.

Correct tests

# tests/test_sample.py
def test_math():
    assert 4 + 1 == 5  # Succès : 5 == 5

After correction and commit, the pipeline displays Tests passed! and the Docker Compose stack shuts down properly thanks to always.

Create a Multibranch Pipeline in Jenkins

1. New Item → azure-voting-app-redis → type Multibranch Pipeline → OK
2. Add a Branch Source → Git
3. Enter the HTTPS URL of the repository and the credentials
4. Do not specify a particular branch — Jenkins will discover all branches automatically
5. Leave the Script Path to Jenkinsfile
6. Save

Jenkins then scans all branches of the repository and creates a subproject for each. In the interface, we see for example:

• main — success
• feature/add-tests — failed Run Tests stage (expected)

3.5 Module 3 Summary

This module covered:

• Configuring a Multibranch Pipeline to manage multiple branches
• Added post steps (success, failure, always)
• Scope of post actions: on a single stage or on the entire pipeline

4. Intégrer la sécurité et la conformité des conteneurs

4.1 Introduction

This module covers container security and compliance integration. The plan:

1. Upload the container to a registry — necessary so that scanners can access it
2. Script blocks — allow Groovy code to be executed inline in the pipeline
3. Docker Pipeline advanced steps — manipulation of registries, credentials, etc.
4. Scanning tools: Grype (Anchore) and Clair
5. Parallel stages — run multiple scans simultaneously to optimize build times

In the progress of the course, we are ¾ of the way through the pipeline: the Docker container is built and pushed, then analyzed in parallel by two tools.

4.2 Demo: Pushing a Docker container to a registry

The Jenkinsfile with script block and Docker Push

pipeline {
    agent any
    stages {
        // ... stages précédents ...
        stage('Docker Push') {
            steps {
                echo "Running in $WORKSPACE"
                dir("$WORKSPACE/azure-vote") {
                    script {
                        docker.withRegistry('', 'dockerhub') {
                            def image = docker.build("blackdentech/jenkins-course:2023")
                            image.push()
                        }
                    }
                }
            }
        }
    }
}

Detailed explanations:

• $WORKSPACE: Built-in Jenkins environment variable indicating the build working directory
• dir("$WORKSPACE/azure-vote"): Change directory step — like a cd in bash. Everything inside will run in this directory. Here, we point to azure-vote because this is where the Dockerfile is located
• script { ... }: Block allowing you to write pure Groovy code in the pipeline. Essential for certain operations that are not available as declarative steps
• docker.withRegistry('', 'dockerhub'): Docker Pipeline plugin method
• First parameter: Registry URL (empty = Docker Hub by default)
• Second parameter: ID of Jenkins credentials for this registry
• def image = docker.build("blackdentech/jenkins-course:2023"): Builds the image with the specified tag
• image.push(): Pushes the image to the registry

Configure Docker Hub credentials in Jenkins

Docker Hub credentials must be added in Jenkins before running the pipeline:

1. Credentials (left menu) → Global credentials
2. Add credentials
3. Type: Username with password
4. Username: Docker Hub user name
5. Password: password or access token Docker Hub
6. ID: dockerhub (must exactly match the ID in the Jenkinsfile)

Important: The ID of the credentials in Jenkins must match exactly what is specified in the Jenkinsfile ('dockerhub'). Jenkins will never allow the password to be seen once saved — credentials are stored securely and will only be used by pipeline stages.

Result in Docker Hub

After successful execution, the image blackdentech/jenkins-course:2023 appears in the Docker Hub registry. The Jenkins log shows:

• docker login succeeded with URL index.docker.io
• Image construction
• Pushing the image to the registry

4.3 Container Scanner Clear

Clair is an open-source project maintained by the Quay.io team. This is a static code analysis tool.

Features

• Open-source, maintained by Quay.io / Red Hat
• Static vulnerability scanning tool for containers
• Integrates well with CoreOS or Red Hat environments
• Very “enterprise” in its approach

Resources

• GitHub Repository: github.com/quay/clair
• Documentation: available on the project website
• Support: Mailing List, IRC channel, bug report

Architecture of Clair (for a home lab)

For a laboratory environment, Clair requires:

• A PostgreSQL server (CVE database)
• A Clear server itself
• A clear-scanner utility for the CLI interface

In an enterprise environment with Quay.io or CoreOS, these components are typically hosted on dedicated servers.

4.4 Demo: Clear

Jenkins multi-agent architecture

The course uses two build agents (build-1 and build-2), each with specific tools installed. It is essential to ensure that the stages are running on the correct agent, otherwise the pipeline fails because the tools are not available.

• build-2: Has Clear tools installed

To see the agents: Manage Jenkins → Nodes

How clear-scanner works on the command line

# Démarrer PostgreSQL pour Clair
docker run -d --name clair-db postgres:latest

# Démarrer le serveur Clair
docker run -d --name clair \
    --link clair-db:postgres \
    quay.io/coreos/clair:latest

# D'abord, récupérer l'image à analyser
docker pull blackdentech/jenkins-course:2023

# Lancer le scan avec clair-scanner
./clair-scanner \
    --ip <ip_interface_docker_zero> \
    blackdentech/jenkins-course:2023

Result on the demo application: A very large number of vulnerabilities were found — so many that they don’t even fit in the terminal buffer. CVEs are listed and can be categorized and addressed. In a demo context, we can ignore them, but in production, it is essential to follow them and correct them.

Integrate Clear into the Jenkinsfile

stage('Run Clair') {
    steps {
        sh script: """
            docker run -d --name clair-db postgres:latest
            docker run -d --name clair \
                --link clair-db:postgres \
                quay.io/coreos/clair:latest
        """
    }
}
stage('Run Clair scan') {
    steps {
        sh script: """
            docker pull blackdentech/jenkins-course:2023
            ./clair-scanner \
                --ip <docker_zero_ip> \
                blackdentech/jenkins-course:2023
        """
    }
}

These courses use Shell Script (sh) steps to execute commands, consistent with what was learned in previous modules.

Result in Jenkins: The job fails because many vulnerabilities are found, which is the expected behavior. The logs show all CVEs found with their references, allowing teams to process them.

4.5 Grype Container Scanner (Anchore)

Grype is the modern replacement for the (now retired) Anchore Engine. It is maintained by the Anchore team.

Features

• Open-source with enterprise features
• Vulnerability and policy compliance scanning tool
• Has a dedicated Jenkins plugin (GrypeScanner Plugin)
• Compatible with Warnings Next Generation Plugin for visual reporting

Installing the Grype plugin

1. Manage Jenkins → Plugins → Available plugins
2. Search for Grype
3. Install GrypeScanner Plugin
4. Optionally, install Warnings Next Generation Plugin for trend charts

Usage difference

Unlike Clair which is called via shell scripts, Grype has a dedicated Jenkins step thanks to its plugin.

4.6 Demo: Grype and parallel courses

Concept: Optimization of build times with parallel courses

Problem: If we have three sequential stages of 5 minutes each, the total pipeline lasts 15 minutes.

Solution: If the stages are not interdependent, they can be executed in parallel:

[Stage A - 5min] → [Stage B - 5min]  → Total : 10 min
                   [Stage C - 5min]  /

Instead of 15 minutes sequentially, the pipeline now only takes 10 minutes.

Significant constraint

If the stages are interdependent, running them in parallel can cause problems. Example: If Grype searches for the Docker image in the registry while the push stage is still in progress, the scan may fail because the image is not yet available.

Solution adopted: Ensure that the build and push stages run before the scanning stages, then launch the two scans in parallel with each other.

Build → Push → [Clair scan  ]  ← en parallèle
               [Grype scan  ]

Jenkinsfile with parallel stages

pipeline {
    agent any
    stages {
        // ... stages précédents (build, push) ...
        stage('Container Scanning') {
            parallel {
                stage('Run Clair scan') {
                    agent { label 'build-2' }
                    steps {
                        // Clair s'exécute sur build-2
                        // Commenté car trop de vulnérabilités en démo :
                        // sh script: "./clair-scanner ..."
                        sleep(time: 1, unit: 'MINUTES')
                    }
                }
                stage('Run Grype') {
                    agent { label 'build-1' }
                    steps {
                        sh script: """
                            grype blackdentech/jenkins-course:2023
                        """
                    }
                    post {
                        always {
                            recordIssues(tools: [grype()])
                        }
                    }
                }
            }
        }
    }
}

Key points:

• parallel { ... }: Block which contains stages to be executed in parallel
• agent { label 'build-2' }: Specifies that this stage should run on the agent labeled build-2. The global agent of the pipeline is any, but we can specify the agent at the stage level
• recordIssues(tools: [grype()]): Step of the Warnings Next Generation Plugin to aggregate results and generate trend graphs

Configuring agent labels in Jenkins

1. Manage Jenkins → Nodes → Click on an agent → Configure
2. In the Labels field, add the desired labels (e.g.: build-1, docker)
3. An agent can have several labels
4. If we use docker as label instead of build-1, the stage can run on any agent with this label

Reading training logs in parallel

When stages run in parallel, Jenkins annotates each log line with the name of the source stage, allowing you to distinguish which output comes from which stage. Ex. :

[Run Clair scan] Sleeping for 1 minute and 0 seconds
[Run Grype] Vulnerability found: CVE-2023-XXXX
[Run Grype] ...

Performance Results

In the course example:

• Clear scan (simulated with sleep): exactly 1 minute
• Grype scan: approximately 2 minutes (dominant)
• Total in parallel: ~3 minutes instead of ~3 minutes sequentially (with Clear time added)

In production with a true Clear scan, the gains would be much more significant.

4.7 Summary of Module 4

This module covered:

• Uploading containers to a registry and using script blocks
• Configuring and using the Grype plugin (GrypeScanner)
• Configuring and using the Clair tool via shell scripts
• Running stages in parallel to optimize build times

5. Implement continuous deployment pipelines

5.1 Introduction

This module covers the implementation of continuous deployment (CD) pipelines. We are at the end of the pipeline:

1. Deploy to two environments: QA and production (PROD)
2. Deploy to QA: automatically on each push to the main branch
3. Deploy to PROD: only from the main branch and with manual approval
4. when conditions: control when a step executes
5. input steps: allow manual approval before continuing

Recommended Kubernetes resources

The course does not claim to be exhaustive on Kubernetes. Suggested resources to learn more:

• Getting Started with Kubernetes
• Kubernetes for Developers: Core Concepts

5.2 When conditions and input steps

The when condition

In a Multibranch Pipeline with several branches (main, feature, bugfix…), certain steps should only be executed on specific branches. Example:

• Build: must run on all branches
• Tests: must run on all branches
• Deploy: must run only on the main branch

We don’t want to deploy every feature branch or bugfix — potentially non-functional code — into production.

Syntax of the when condition

pipeline {
    agent any
    stages {
        stage('Hello World') {
            steps {
                echo 'Hello World'  // S'exécute toujours
            }
        }
        stage('Runs on main') {
            when {
                branch 'main'
            }
            steps {
                echo 'Hello from the main branch!'
            }
        }
    }
}

Available when condition types

The when condition is not limited to the branch type. We can also use:

• branch: Condition on the branch name
• environment: Condition on an environment variable
• Git release variable: Git version tags (e.g.: v1.0.0)
• Custom Groovy expression: Any Groovy assertion that returns true

The input step

The input step allows you to require manual approval before continuing the pipeline. Typical use cases:

• A UX or security team must validate the application in QA before deployment in PROD
• Wait for an opportune moment (e.g.: no production interruption during an important event)
• Comply with an audit or compliance process

stage('Approve Deploy to PROD') {
    steps {
        input message: 'Deploy to PROD?'
    }
    options {
        timeout(time: 1, unit: 'HOURS')
    }
}

Behavior:

• Pipeline waits until an authorized user responds
• Options available in the Jenkins interface: Proceed (continue) or Abort (cancel)
• If the user clicks Abort: the pipeline stops with the status ABORTED
• If the user clicks Proceed: the pipeline continues to the next stage

Advanced input options

The Jenkins Pipeline documentation provides additional options not covered in the course:

• Custom drop-down lists: Multiple options for users
• Restriction by group: Limit who can respond to the input step (integration with the Identity Provider)

Timeout on input steps

To prevent the pipeline from being blocked indefinitely (costly on hourly billed cloud agents):

options {
    timeout(time: 1, unit: 'HOURS')
}

After 1 hour without response, the step times out automatically.

5.3 Demo: When conditions and input steps

Retrieve Kubernetes credentials (AKS — Azure)

# Dans Azure Cloud Shell
az aks get-credentials \
    --resource-group <nom-du-resource-group> \
    --name prod-demo-cluster

This command generates a kubeconfig file in ~/.kube/config. This file contains the credentials to connect to the cluster. Sensitive values ​​are hidden in the display.

Note: This command is specific to Azure. Each cloud provider has its own method. Once the kubeconfig file is obtained, the deployment is identical regardless of the platform.

Import kubeconfig into Jenkins

1. Credentials in the Jenkins menu → Global or domain specific to the Multibranch Pipeline
2. Add credentials
3. Type: Secret file (for a kubeconfig file)
4. Upload the kubeconfig file
5. Give an ID: qa-kubeconfig or prod-kubeconfig

In the course, two kubeconfig files are imported:

• qa-kubeconfig — for QA cluster
• prod-kubeconfig — for the PROD cluster (scoped to the Multibranch Pipeline to limit access)

Full Jenkinsfile with QA and PROD deployment

pipeline {
    agent any
    stages {
        // ... stages précédents (build, test, push, scan) ...
        
        stage('QA Deploy') {
            when {
                branch 'feature/k8s-deploy'  // En production : utiliser 'main'
            }
            environment {
                KUBECONFIG = credentials('qa-kubeconfig')
            }
            steps {
                sh script: """
                    kubectl apply \
                        -f azure-vote-all-in-one-redis.yaml \
                        --kubeconfig=$KUBECONFIG
                """
            }
        }
        
        stage('Approve Deploy to PROD') {
            when {
                branch 'feature/k8s-deploy'  // En production : utiliser 'main'
            }
            options {
                timeout(time: 1, unit: 'HOURS')
            }
            steps {
                input message: 'Deploy to PROD?'
            }
        }
        
        stage('PROD Deploy') {
            when {
                branch 'feature/k8s-deploy'  // En production : utiliser 'main'
            }
            environment {
                KUBECONFIG = credentials('prod-kubeconfig')
            }
            steps {
                sh script: """
                    kubectl apply \
                        -f azure-vote-all-in-one-redis.yaml \
                        --kubeconfig=$KUBECONFIG
                """
            }
        }
    }
}

Key points:

• environment { KUBECONFIG = credentials('qa-kubeconfig') }: Creates an environment variable by injecting the contents of the Secret file. This variable is scoped to the stage, not the entire pipeline — which ensures that each stage uses the right kubeconfig for the right cluster
• kubectl apply -f azure-vote-all-in-one-redis.yaml --kubeconfig=$KUBECONFIG: Deploys the Kubernetes manifest using the specified kubeconfig. The kubectl apply syntax is universal regardless of the platform
• The environment block is scoped at the stage level to ensure the isolation of QA vs PROD credentials
• The DRY (Don’t Repeat Yourself) principle is respected — QA Deploy and PROD Deploy have essentially the same code, with different credentials

Behavior of input step in Jenkins

When running the pipeline:

1. QA Deploy stages run automatically
2. The pipeline stops at the Approve Deploy to PROD stage
3. In the Jenkins interface, a Proceed button and an Abort button appear
4. If Abort is clicked: the pipeline ends with the status ABORTED (the PROD Deploy stage is marked as “skipped due to earlier failure(s)”)
5. If Proceed is clicked: the PROD Deploy stage runs and deploys to the production cluster

Deployment result

Jenkins log shows that kubectl apply deployed Kubernetes resources (services, deployments) to the target cluster. The services are created/updated, and the two clusters (QA and PROD) are isolated by their respective kubeconfig.

5.4 Summary of Module 5

This module covered:

• Using when conditions to control the execution of stages
• Using input steps for manual approval before deployment
• Deploying to Kubernetes (AKS) from Jenkins via kubeconfig
• Approach is cloud agnostic — only kubeconfig retrieval is platform specific

6. Troubleshooting and Improving Jenkins Pipelines

6.1 Introduction

This final module covers troubleshooting and improving Jenkins pipelines. There’s little left to cover on the pipeline itself — we’re focusing on how to refine it with:

• The Groovy Shared Libraries: Import entire pipelines and shared functions

6.2 Demo: The Declarative Directive Generator

Integrated Jenkins tools for development support

In the view of a Jenkins pipeline, clicking on Pipeline Syntax (left menu) gives access to several tools:

1. Snippet Generator: Generates example steps for the pipeline (seen in previous modules)
2. Global Variables Reference: Lists all variables available in the pipeline — environment variables, scripts (including docker, used in module 3)
3. Declarative Directive Generator: Generates pipeline directives (agents, when conditions, options, stages, etc.)

The Declarative Directive Generator

Just like the Snippet Generator generates steps, the Declarative Directive Generator generates complex pipeline blocks. It is particularly useful for:

• Generate complicated when conditions
• Configure agents with different options (Docker, labels, etc.)
• Generate options, triggers, matrix, environment, agent blocks

Usage example: Generate a stage with a Docker agent to run Golang

In the Declarative Directive Generator:

1. Choose Internship from the list
2. Name the course: Run Golang
3. Choose to add an Agent → type Docker
4. Specify image: golang:latest
5. Click on Generate

Result generated:

stage('Run Golang') {
    agent {
        docker {
            image 'golang:latest'
        }
    }
    steps {
        sh 'go version'
    }
}

This code can be copied and pasted directly into a Jenkinsfile.

Demonstration with two stages (Golang installed via Docker vs. not installed)

pipeline {
    agent any
    stages {
        stage('Run Golang') {
            agent {
                docker {
                    image 'golang:latest'
                }
            }
            steps {
                sh 'go version'
            }
        }
        stage('Fail Golang') {
            steps {
                sh 'go version'  // Échoue car go n'est pas installé sur l'agent
            }
        }
    }
}

Result:

• Run Golang: SUCCESS — Jenkins downloads the golang:latest image, runs it as a container on the agent, and launches go version inside. The version shown is go1.21.4 (at time of saving)
• Fail Golang: FAILED — go: command not found because Go is not installed on the physical agent

This perfectly illustrates how you can run tools not installed on agents by running them in Docker containers.

6.3 Jenkins Shared Libraries

Problem resolved by Shared Libraries

So far, all pipeline code is in a single repository. But there are situations where this is not suitable:

• Don’t want developers to have access to pipeline code
• Want to provide developers with a set of reusable functions/utilities
• Have a shared architecture requiring all applications to be deployed with the same pipelines

What is a Shared Library?

A Jenkins Shared Library allows pipeline code and several utilities to be moved outside of the application repository, and referenced from the application repository.

Advantages:

• Scalability: Maintain the same pipeline code for multiple applications without duplicating
• Isolation: Separate responsibilities between application code and deployment code
• Reusability: Shared functions accessible from all repositories
• Centralization: One place to maintain deployment patterns

Structure of a Shared Library

A Shared Library repository must have either an src/ directory or a vars/ directory:

demo-shared-pipeline/
└── vars/
    ├── helloWorld.groovy
    ├── helloWorld.txt      ← Documentation
    ├── helloArgs.groovy
    ├── helloArgs.txt       ← Documentation
    ├── echoPipeline.groovy
    └── echoPipeline.txt    ← Documentation

Each .groovy file can have an associated .txt file for documentation. These .txt files describe what the script does.

Important naming rule

The name of the .groovy file (without extension) must match exactly the name used in the Jenkinsfile to call it.

6.4 Demo: Basic Groovy functions in a shared library

File helloWorld.groovy

def call() {
    println 'Hello World!'
}

Important points:

• The call function is the default function — it is what is executed when the script is called
• println in Groovy/Java is equivalent to print or echo

Use in a Jenkinsfile

@Library('demo-shared-pipeline') _

pipeline {
    agent any
    stages {
        stage('Call Library Function') {
            steps {
                script {
                    helloWorld()
                }
            }
        }
    }
}

Explanations:

• @Library('demo-shared-pipeline') _: Shared Library import decorator. Points to the GitHub repository demo-shared-pipeline. The _ is required by Jenkins syntax for Shared Libraries
• script { ... }: Groovy block needed to call Shared Library functions
• helloWorld(): Calls the helloWorld.groovy file and executes its default call function

Note: For a private repository, the Jenkins documentation explains how to reference a private Shared Library with credentials.

Result in Jenkins

The Output Console displays: Hello World!

This message does not come from the code in the application repository, but from a shared utility in the Shared Library. This approach is ideal for centralizing reusable functions.

6.5 Demo: Advanced Groovy Features

File helloArgs.groovy

def call(String name) {
    println "Hello ${name}"
}

def goodbyeWorld(String name) {
    println "Goodbye ${name}"
}

Important points:

• The call function takes a String type parameter named name
• ${name} is interpolating Groovy variables into a string
• goodbyeWorld is a second function in the same file, with a different name than call
• To call a function not named call, we use the notation FileName.FunctionName()

Use in a Jenkinsfile

@Library('demo-shared-pipeline') _

pipeline {
    agent any
    stages {
        stage('Hello') {
            steps {
                script {
                    helloArgs('Jenkins')  // Appelle la fonction call()
                }
            }
        }
        stage('Goodbye') {
            steps {
                script {
                    helloArgs.goodbyeWorld('Jenkins')  // Appelle goodbyeWorld()
                }
            }
        }
    }
}

Result:

• Stage Hello: Hello Jenkins
• Goodbye course: Goodbye Jenkins

Parameter 'Jenkins' is passed from the Jenkinsfile — it is not defined in the Shared Library. This is the power of parameterized functions.

Concrete use cases

Shared Libraries with arguments are ideal for:

• Authentication scripts: Pass different configurations depending on the environment
• Slack notifications: Send messages with dynamic information in case of failure
• Deployment functions: Pass environment name, URL, etc.

6.6 Demo: Complete pipelines in a shared library

File echoPipeline.groovy

def call(body) {
    def config = [:]
    body.resolveStrategy = Closure.DELEGATE_FIRST
    body.delegate = config
    body()

    pipeline {
        agent any
        stages {
            stage('echo from the other side') {
                steps {
                    echo "${config.message}"
                }
            }
        }
    }
}

Explanations:

• The call function takes a body — a closure which will contain the parameters to pass
• The configuration lines allow you to delegate the contents of body to the Map config
• From the pipeline { ... } line, this is a standard Jenkins pipeline — identical to those already seen in the course
• ${config.message} accesses the message parameter passed via the configuration block

Use in a Jenkinsfile

Instead of having a pipeline { ... } block in the Jenkinsfile, we directly call the Shared Library pipeline:

@Library('demo-shared-pipeline') _

echoPipeline {
    message = 'I tried to ping a thousand times'
}

Key points:

• No pipeline { ... } block in the Jenkinsfile — it is defined in the Shared Library
• Syntax resembles declarative configuration
• Parameter message is passed via configuration block
• This approach allows for fully configurable and centralized pipelines

Result in Jenkins

The Output Console displays the echo from the other side stage with the step:

I tried to ping a thousand times

The pipeline was found in the Shared Library, the echo from the other side stage was executed with the message parameter passed from the Jenkinsfile.

Use cases for pipelines in Shared Libraries

This approach is particularly powerful in shared architectures:

• Enterprise Standard Architecture: All microservices use the same deployment pipeline — a single source of truth to maintain
• Pipeline templates: Different types of applications (API, services, frontends) each have their own pipeline template
• Governance: The DevOps team controls the pipeline without touching the application repositories

6.7 Summary of module 6

This module covered:

• Internal Jenkins tools available: Declarative Directive Generator and Global Variables Reference
• Using Groovy Shared Libraries for reusable pipelines and methods
• Basic (call) and advanced functions (named functions, parameters)
• Entire pipelines hosted in a Shared Library

7. Appendix: Complete Code Examples

Jenkinsfile Hello World

pipeline {
    agent any
    stages {
        stage('Hello') {
            steps {
                echo 'Hello World'
            }
        }
    }
}

Jenkinsfile with Docker build

pipeline {
    agent any
    stages {
        stage('Verify Branch') {
            steps {
                echo "$GIT_BRANCH"
            }
        }
        stage('Docker Build') {
            steps {
                sh script: """
                    docker compose build
                """
            }
        }
    }
}

Jenkinsfile with unit tests and post blocks

pipeline {
    agent any
    stages {
        stage('Verify Branch') {
            steps {
                echo "$GIT_BRANCH"
            }
        }
        stage('Docker Build') {
            steps {
                sh script: "docker compose build"
            }
        }
        stage('Start App') {
            steps {
                sh script: "docker compose up -d"
            }
        }
        stage('Run Tests') {
            steps {
                sh script: "pytest tests/"
            }
            post {
                success {
                    echo 'Tests passed!'
                }
                failure {
                    echo 'Tests failed'
                }
            }
        }
    }
    post {
        always {
            sh script: "docker compose down"
        }
    }
}

Jenkinsfile with Docker Push (script block)

pipeline {
    agent any
    stages {
        stage('Docker Push') {
            steps {
                echo "Running in $WORKSPACE"
                dir("$WORKSPACE/azure-vote") {
                    script {
                        docker.withRegistry('', 'dockerhub') {
                            def image = docker.build("blackdentech/jenkins-course:2023")
                            image.push()
                        }
                    }
                }
            }
        }
    }
}

Jenkinsfile with parallel container scanning

pipeline {
    agent any
    stages {
        stage('Container Scanning') {
            parallel {
                stage('Run Clair scan') {
                    agent { label 'build-2' }
                    steps {
                        sh script: """
                            docker run -d --name clair-db postgres:latest
                            docker run -d --name clair \
                                --link clair-db:postgres \
                                quay.io/coreos/clair:latest
                            docker pull blackdentech/jenkins-course:2023
                            ./clair-scanner --ip <docker_zero_ip> \
                                blackdentech/jenkins-course:2023
                        """
                    }
                }
                stage('Run Grype') {
                    agent { label 'build-1' }
                    steps {
                        sh script: "grype blackdentech/jenkins-course:2023"
                    }
                    post {
                        always {
                            recordIssues(tools: [grype()])
                        }
                    }
                }
            }
        }
    }
}

Jenkinsfile with Kubernetes QA and PROD deployment

pipeline {
    agent any
    stages {
        stage('QA Deploy') {
            when {
                branch 'main'
            }
            environment {
                KUBECONFIG = credentials('qa-kubeconfig')
            }
            steps {
                sh script: """
                    kubectl apply \
                        -f azure-vote-all-in-one-redis.yaml \
                        --kubeconfig=$KUBECONFIG
                """
            }
        }
        stage('Approve Deploy to PROD') {
            when {
                branch 'main'
            }
            options {
                timeout(time: 1, unit: 'HOURS')
            }
            steps {
                input message: 'Deploy to PROD?'
            }
        }
        stage('PROD Deploy') {
            when {
                branch 'main'
            }
            environment {
                KUBECONFIG = credentials('prod-kubeconfig')
            }
            steps {
                sh script: """
                    kubectl apply \
                        -f azure-vote-all-in-one-redis.yaml \
                        --kubeconfig=$KUBECONFIG
                """
            }
        }
    }
}

Jenkinsfile with Docker agent for Golang

pipeline {
    agent any
    stages {
        stage('Run Golang') {
            agent {
                docker {
                    image 'golang:latest'
                }
            }
            steps {
                sh 'go version'
            }
        }
        stage('Fail Golang') {
            // Pas d'agent Docker — go n'est pas installé sur l'agent
            steps {
                sh 'go version'  // Échoue : go not found
            }
        }
    }
}

Shared Library — helloWorld.groovy

// vars/helloWorld.groovy
def call() {
    println 'Hello World!'
}

Shared Library — helloArgs.groovy

// vars/helloArgs.groovy
def call(String name) {
    println "Hello ${name}"
}

def goodbyeWorld(String name) {
    println "Goodbye ${name}"
}

Shared Library — echoPipeline.groovy

// vars/echoPipeline.groovy
def call(body) {
    def config = [:]
    body.resolveStrategy = Closure.DELEGATE_FIRST
    body.delegate = config
    body()

    pipeline {
        agent any
        stages {
            stage('echo from the other side') {
                steps {
                    echo "${config.message}"
                }
            }
        }
    }
}

Jenkinsfile using helloWorld from a Shared Library

@Library('demo-shared-pipeline') _

pipeline {
    agent any
    stages {
        stage('Call Library Function') {
            steps {
                script {
                    helloWorld()
                }
            }
        }
    }
}

Jenkinsfile using helloArgs from a Shared Library

@Library('demo-shared-pipeline') _

pipeline {
    agent any
    stages {
        stage('Hello') {
            steps {
                script {
                    helloArgs('Jenkins')
                }
            }
        }
        stage('Goodbye') {
            steps {
                script {
                    helloArgs.goodbyeWorld('Jenkins')
                }
            }
        }
    }
}

Jenkinsfile calling a complete pipeline from a Shared Library

@Library('demo-shared-pipeline') _

echoPipeline {
    message = 'I tried to ping a thousand times'
}

8. Appendix: Quick Reference of Concepts

Search Terms

modern · ci · cd · pipeline · jenkins · ci/cd · git · devops · jenkinsfile · shared · library · docker · pipelines · input · container · parallel · unit · deployment · scripted · tests · clear · condition · configure · features