What is Pipeline

In Jenkins, a Pipeline is a suite of plugins that allows you to define and automate the continuous integration and continuous delivery (CI/CD) process in a scripted format. It helps manage complex workflows by defining the build, test, and deployment stages as code.

Types of Pipelines in Jenkins

1. Declarative Pipeline

   • Uses a structured, easy-to-read syntax.

   • Defined inside a pipeline {} block.

   • Example:

       pipeline {
           agent any
           stages {
               stage('Build') {
                   steps {
                       echo 'Building the application...'
                   }
               }
               stage('Test') {
                   steps {
                       echo 'Running tests...'
                   }
               }
               stage('Deploy') {
                   steps {
                       echo 'Deploying the application...'
                   }
               }
           }
       }
     

2. Scripted Pipeline

   • More flexible but complex.

   • Uses Groovy scripting.

   • Defined inside a node {} block.

   • Example:

       node {
           stage('Build') {
               echo 'Building the application...'
           }
           stage('Test') {
               echo 'Running tests...'
           }
           stage('Deploy') {
               echo 'Deploying the application...'
           }
       }
     

Key Features of Jenkins Pipeline

• Pipeline as Code: Defined in a Jenkinsfile, making it version-controlled.

• Stages and Steps: Helps in structuring workflows into different phases.

• Parallel Execution: Enables running tasks concurrently.

• Integration with DevOps Tools: Works with Git, Docker, Kubernetes, and other tools.

Why You Should Have a Pipeline?

A Jenkins Pipeline is essential for automating software development workflows, particularly in CI/CD (Continuous Integration/Continuous Deployment). Here are the key reasons why using a pipeline is beneficial:

1. Automation and Efficiency

• Eliminates the need for manual execution of build, test, and deployment steps.

• Ensures consistency across different environments.

2. Pipeline as Code

• Pipelines are defined in a Jenkinsfile, allowing version control and better collaboration.

• Enables teams to share and maintain CI/CD workflows efficiently.

3. Faster and Reliable Deployments

• Automates testing and validation, reducing errors in production.

• Supports rollback mechanisms in case of deployment failures.

4. Scalability and Parallel Execution

• Allows running multiple tasks in parallel, reducing overall build time.

• Supports distributed builds across different agents.

5. Integration with DevOps Tools

• Works seamlessly with Git, Docker, Kubernetes, AWS, Terraform, Ansible, and other DevOps tools.

• Supports a wide range of plugins to extend functionality.

6. Better Visibility and Monitoring

• Provides a clear visual representation of different pipeline stages.

• Offers real-time logs and detailed insights for troubleshooting.

7. Customization and Flexibility

• Allows choosing between Declarative and Scripted Pipelines based on complexity and project requirements.

• Supports conditional execution, custom scripts, and advanced configurations.

Real-World Use Case: CI/CD for a Web Application

Consider a scenario where a development team is working on a web application. A Jenkins Pipeline can:

1. Fetch the latest code from a version control system like GitHub.

2. Build and compile the application.

3. Run automated tests, including unit and integration tests.

4. Deploy the application to a staging environment for further validation.

5. Deploy to production if all tests pass successfully.

Pipeline syntax

Jenkins Pipeline syntax defines how CI/CD workflows are written and executed. Jenkins supports two types of pipelines: Declarative and Scripted.

---

1. Declarative Pipeline Syntax

Declarative Pipeline provides a more structured and easy-to-read syntax. It is defined within a pipeline {} block.

Basic Structure

pipeline {
    agent any
    stages {
        stage('Build') {
            steps {
                echo 'Building the application...'
            }
        }
        stage('Test') {
            steps {
                echo 'Running tests...'
            }
        }
        stage('Deploy') {
            steps {
                echo 'Deploying the application...'
            }
        }
    }
}

Key Directives

• pipeline {}: Defines the pipeline structure.

• agent: Specifies where the pipeline should run (any, none, or a specific agent).

• stages {}: Contains multiple stages in the pipeline.

• stage('Name') {}: Defines each step in the workflow.

• steps {}: Contains the commands to execute.

Additional Features

• Post Actions (execute actions based on pipeline result):

    pipeline {
        agent any
        stages {
            stage('Build') {
                steps {
                    echo 'Building...'
                }
            }
        }
        post {
            success {
                echo 'Pipeline executed successfully!'
            }
            failure {
                echo 'Pipeline failed!'
            }
        }
    }
  

• Parallel Execution (run multiple tasks at the same time):

    pipeline {
        agent any
        stages {
            stage('Parallel Tasks') {
                parallel {
                    stage('Task A') {
                        steps { echo 'Running Task A' }
                    }
                    stage('Task B') {
                        steps { echo 'Running Task B' }
                    }
                }
            }
        }
    }
  

---

2. Scripted Pipeline Syntax

The Scripted Pipeline provides more flexibility using Groovy scripting but is less structured. It is defined inside a node {} block.

Basic Structure

node {
    stage('Build') {
        echo 'Building the application...'
    }
    stage('Test') {
        echo 'Running tests...'
    }
    stage('Deploy') {
        echo 'Deploying the application...'
    }
}

Key Differences from Declarative Pipelines

• Uses Groovy scripting for more control and flexibility.

• Requires manual error handling.

• Recommended for complex workflows.

Example with Conditional Execution

node {
    stage('Build') {
        echo 'Building...'
    }
    stage('Test') {
        echo 'Running tests...'
        if (currentBuild.result == 'FAILURE') {
            error 'Stopping pipeline due to test failure'
        }
    }
    stage('Deploy') {
        echo 'Deploying...'
    }
}

---

Choosing the Right Pipeline Syntax

For most CI/CD workflows, a Declarative Pipeline is recommended due to its simplicity. However, for highly dynamic workflows, Scripted Pipeline offers greater flexibility.

Task-01

• Create a New Job, this time select Pipeline instead of Freestyle Project.

• Follow the Official Jenkins Hello world example

• Complete the example using the Declarative pipeline.

This task involves creating a new Pipeline Job in Jenkins and following the official Hello World example using a Declarative Pipeline.

---

Step 1: Create a New Pipeline Job

1. Open Jenkins Dashboard.

2. Click on "New Item".

3. Enter a name for your job (e.g., Hello-World-Pipeline).

4. Select "Pipeline" (instead of Freestyle Project).

5. Click OK.

---

Step 2: Define the Pipeline (Using Declarative Syntax)

1. Scroll down to the Pipeline section.

2. Choose Pipeline script.

3. Copy and paste the following Hello World Declarative Pipeline code:

pipeline {
    agent any
    stages {
        stage('Hello') {
            steps {
                echo 'Hello, World!'
            }
        }
    }
}

---

Step 3: Save and Run the Pipeline

1. Click Save.

2. Click Build Now.

3. Navigate to Build History and click on the latest build.

4. Click Console Output to verify that it prints:

    Hello, World!
   

---

Explanation of the Pipeline Code

• pipeline {}: Defines the pipeline block.

• agent any: Runs the pipeline on any available agent.

• stages {}: Contains one or more stages.

• stage('Hello') {}: Defines a stage named "Hello".

• steps {}: Contains the commands to be executed.

• echo 'Hello, World!': Prints "Hello, World!" in the console.

Jenkins Declarative Pipeline with Docker

Jenkins integrates seamlessly with Docker to run builds inside Docker containers. Using a Declarative Pipeline, you can define and execute your CI/CD process within a containerized environment.

---

1. Prerequisites

Before running a Jenkins pipeline with Docker, ensure:

• Jenkins has the Docker plugin installed.

• Docker is installed and running on the Jenkins agent.

• The Jenkins user has permission to run Docker commands (docker group).

---

2. Basic Declarative Pipeline Using Docker

This pipeline runs inside a Docker container and executes a simple script.

pipeline {
    agent {
        docker {
            image 'python:3.9'
            args '--rm'
        }
    }
    stages {
        stage('Run Python') {
            steps {
                sh 'python --version'
                sh 'echo "Hello from Docker Container!"'
            }
        }
    }
}

Explanation

• agent { docker { image 'python:3.9' } }: Runs the pipeline inside a python:3.9 Docker container.

• args '--rm': Removes the container after execution.

• sh 'python --version': Executes a shell command inside the container.

• sh 'echo "Hello from Docker Container!"': Prints a message.

---

3. Using Docker to Build and Run an Application

This pipeline pulls a Docker image, builds a sample application, and runs it inside a container.

pipeline {
    agent any
    environment {
        IMAGE_NAME = 'my-app:latest'
    }
    stages {
        stage('Clone Repository') {
            steps {
                git 'https://github.com/example/repo.git'
            }
        }
        stage('Build Docker Image') {
            steps {
                script {
                    sh 'docker build -t $IMAGE_NAME .'
                }
            }
        }
        stage('Run Container') {
            steps {
                script {
                    sh 'docker run -d --name my-container -p 8080:80 $IMAGE_NAME'
                }
            }
        }
    }
    post {
        always {
            sh 'docker stop my-container || true'
            sh 'docker rm my-container || true'
            sh 'docker rmi $IMAGE_NAME || true'
        }
    }
}

Explanation

• stage('Clone Repository'): Clones the source code from a Git repository.

• stage('Build Docker Image'): Builds a Docker image using the Dockerfile.

• stage('Run Container'): Runs the application inside a Docker container and maps port 8080.

• post { always { ... } }: Ensures cleanup by stopping and removing the container and image.

---

4. Running Jenkins Inside Docker

If you want to run Jenkins itself inside Docker, use the following command:

docker run -d --name jenkins \
    -p 8080:8080 -p 50000:50000 \
    -v jenkins_home:/var/jenkins_home \
    jenkins/jenkins:lts

This allows Jenkins to run as a containerized service.

Task-01

• Create a docker-integrated Jenkins declarative pipeline

• Use the above-given syntax using sh inside the stage block

This task involves setting up a Jenkins Declarative Pipeline that integrates with Docker. The pipeline will:

1. Pull the latest code from a Git repository.

2. Build a Docker image.

3. Run a container from the built image.

4. Clean up the container and image after execution.

---

Step 1: Ensure Prerequisites

Before proceeding, ensure:

• Jenkins has Docker installed and properly configured.

• The Jenkins user has permission to run Docker commands (docker group).

• The required Git repository contains a Dockerfile.

---

Step 2: Create a New Pipeline Job in Jenkins

1. Open Jenkins Dashboard.

2. Click on "New Item".

3. Enter a job name (e.g., Docker-Integrated-Pipeline).

4. Select "Pipeline" and click OK.

5. Scroll down to the Pipeline section and choose Pipeline script.

---

Step 3: Define the Pipeline Script

Copy and paste the following Declarative Pipeline script inside the pipeline editor.

pipeline {
    agent any
    environment {
        IMAGE_NAME = 'my-app:latest'
        CONTAINER_NAME = 'my-container'
        REPO_URL = 'https://github.com/example/repo.git' // Change this to your actual repo
    }
    stages {
        stage('Clone Repository') {
            steps {
                sh 'git clone $REPO_URL app'
            }
        }
        stage('Build Docker Image') {
            steps {
                script {
                    sh 'cd app && docker build -t $IMAGE_NAME .'
                }
            }
        }
        stage('Run Container') {
            steps {
                script {
                    sh 'docker run -d --name $CONTAINER_NAME -p 8080:80 $IMAGE_NAME'
                }
            }
        }
    }
    post {
        always {
            sh 'docker stop $CONTAINER_NAME || true'
            sh 'docker rm $CONTAINER_NAME || true'
            sh 'docker rmi $IMAGE_NAME || true'
        }
    }
}

---

Step 4: Save and Run the Pipeline

1. Click Save.

2. Click Build Now.

3. Monitor the Console Output to verify successful execution.

---

Explanation of the Pipeline

• environment {}: Defines environment variables for Docker image, container name, and Git repository.

• stage('Clone Repository'): Clones the source code repository that contains the Dockerfile.

• stage('Build Docker Image'): Builds the Docker image from the cloned repository.

• stage('Run Container'): Runs the Docker container, exposing port 8080.

• post { always { ... } }: Ensures cleanup by stopping and removing the container and image after execution.

---

Expected Output

• Jenkins pulls the source code.

• Builds a Docker image from the Dockerfile.

• Runs the container exposing port 8080.

• Stops and removes the container and image after execution.

This completes Task-01 by successfully creating a Docker-integrated Jenkins Declarative Pipeline.

Task-02

• Create a docker-integrated Jenkins declarative pipeline using the docker Groovy syntax inside the stage block.

• You won't face errors, you can Follow this documentation

• Complete your previous projects using this Declarative pipeline approach

In this task, you will create a Jenkins Declarative Pipeline that integrates with Docker using the Docker Groovy Syntax inside the stage block. This approach utilizes Jenkins' built-in Docker DSL, making pipeline execution more efficient and readable.

---

Step 1: Ensure Prerequisites

Before proceeding, ensure:

• Jenkins has Docker installed and properly configured.

• The Jenkins user has permission to run Docker commands (docker group).

• The required Git repository contains a Dockerfile.

• The Docker Pipeline Plugin is installed in Jenkins.

---

Step 2: Create a New Pipeline Job in Jenkins

1. Open Jenkins Dashboard.

2. Click on "New Item".

3. Enter a job name (e.g., Docker-Groovy-Pipeline).

4. Select "Pipeline" and click OK.

5. Scroll down to the Pipeline section and choose Pipeline script.

---

Step 3: Define the Pipeline Script

Copy and paste the following Declarative Pipeline script that uses the Docker Groovy syntax:

pipeline {
    agent any
    environment {
        IMAGE_NAME = 'my-app:latest'
        CONTAINER_NAME = 'my-container'
        REPO_URL = 'https://github.com/example/repo.git' // Change this to your actual repo
    }
    stages {
        stage('Clone Repository') {
            steps {
                sh 'git clone $REPO_URL app'
            }
        }
        stage('Build Docker Image') {
            steps {
                script {
                    def appImage = docker.build(IMAGE_NAME, "app")
                }
            }
        }
        stage('Run Container') {
            steps {
                script {
                    def appContainer = docker.image(IMAGE_NAME).run('-d --name ' + CONTAINER_NAME + ' -p 8080:80')
                }
            }
        }
    }
    post {
        always {
            script {
                docker.stop(CONTAINER_NAME)
                docker.removeContainer(CONTAINER_NAME)
                docker.removeImage(IMAGE_NAME)
            }
        }
    }
}

---

Step 4: Save and Run the Pipeline

1. Click Save.

2. Click Build Now.

3. Monitor the Console Output to verify successful execution.

---

Explanation of the Pipeline

• environment {}: Defines environment variables for Docker image, container name, and Git repository.

• stage('Clone Repository'): Clones the source code repository that contains the Dockerfile.

• stage('Build Docker Image'):

  • Uses docker.build(IMAGE_NAME, "app") to build the image from the cloned repository.

• stage('Run Container'):

  • Uses docker.image(IMAGE_NAME).run(...) to start the container.

• post { always { ... } }: Ensures cleanup by stopping and removing the container and image after execution using Docker Groovy syntax.

---

Expected Output

• Jenkins pulls the source code.

• Builds a Docker image using docker.build().

• Runs the container using docker.image().run(), exposing port 8080.

• Stops and removes the container and image after execution.

This completes Task-02 by successfully implementing a Docker-integrated Jenkins Declarative Pipeline using Docker Groovy Syntax.