Vanshika Sharma

Jenkins

Vanshika Sharma's photo
Vanshika Sharma
·

17 min read

Table of contents

What is Jenkins?

Jenkins is an open-source automation server designed to help automate various aspects of software development, primarily building, testing, and deploying applications. It plays a key role in the DevOps lifecycle, making implementing continuous integration (CI) and continuous delivery (CD) pipelines easier.

Key Features of Jenkins:

  1. Automation: Automates repetitive tasks such as code compilation, testing, and deployment.

  2. Extensibility: Supports over 1,800 plugins to integrate with various tools like Git, Docker, Kubernetes, Maven, and more.

  3. Platform Independent: Written in Java, it can run on different operating systems, such as Windows, macOS, and Linux.

  4. Easy Configuration: Provides a web-based interface to configure jobs and pipelines.

  5. Pipeline as Code: Allows defining CI/CD workflows in code using Jenkinsfiles, making pipelines version-controlled.

Benefits of Using Jenkins:

  • Saves time by automating manual steps.

  • Enhances software quality by running automated tests early and frequently.

  • Reduces deployment risks by facilitating faster and incremental releases.

  • Supports distributed builds for scaling large projects.

Typical Use Cases:

  • Building Projects: Compile and build your code automatically.

  • Automated Testing: Run unit, integration, or end-to-end tests on code changes.

  • Continuous Integration: Integrate and validate code from multiple developers regularly.

  • Deployment: Deploy applications to staging or production environments.

Why is Jenkins Necessary?

Efficiency, speed, and reliability are critical to delivering high-quality applications in modern software development. Jenkins addresses these needs by streamlining and automating critical development and deployment processes.

1. Facilitating Continuous Integration (CI)

  • In CI, developers frequently integrate their code changes into a shared repository. Jenkins automates the process of:

    • Detecting changes in the repository.

    • Building and testing the code.

  • Without Jenkins, these tasks would need to be performed manually, increasing the risk of human error and delaying development cycles.

2. Enabling Continuous Delivery (CD)

  • Jenkins helps automate the deployment of validated code to production or staging environments. This ensures faster and more reliable delivery of updates and features to users.

3. Improving Developer Productivity

  • By automating repetitive tasks like builds, tests, and deployments, Jenkins allows developers to focus on writing code instead of managing processes manually.

4. Early Bug Detection

  • Jenkins can automatically run unit tests, integration tests, and static code analysis as soon as new code is committed. This allows bugs to be detected and fixed early, reducing the cost of addressing them later.

5. Scalability and Flexibility

  • Jenkins supports distributed builds across multiple machines, making it highly scalable for projects of any size.

  • With its extensive library of plugins, it integrates seamlessly with tools like Git, Docker, Kubernetes, and more, adapting to diverse workflows.

6. Faster Release Cycles

  • Automating testing and deployment reduces the time required for release cycles, enabling businesses to deliver features to users faster and more reliably.

7. Standardized Workflow

  • Jenkins ensures a consistent build, test, and deployment process across teams, reducing variability and improving software quality.

8. Open Source and Community Support

  • Being open-source, Jenkins is free to use, with a large community contributing plugins, documentation, and troubleshooting support.

Why Can't We Rely Solely on Manual Processes?

Manual processes are:

  • Time-consuming: Repeatedly running tests and deploying changes manually slows down development.

  • Error-prone: Human errors can lead to failed builds, missed bugs, or faulty deployments.

  • Unsustainable at Scale: As the number of developers, features, or users grows, manual workflows become unmanageable.

How to install Jenkins?

1. Prerequisites

Before you install Jenkins:

  • Java Requirement: Jenkins requires Java (8 or 11). Verify Java is installed:

      java -version

    If not installed, install Java:

    • On Ubuntu/Debian:

        sudo apt update
  sudo apt install openjdk-11-jdk

    • On CentOS/Red Hat:

        sudo yum install java-11-openjdk

2. Install Jenkins on Linux (Ubuntu/Debian)

Step 1: Add Jenkins Repository

Run the following commands:

wget -q -O - https://pkg.jenkins.io/debian/jenkins.io.key | sudo apt-key add -
sudo sh -c 'echo deb http://pkg.jenkins.io/debian-stable binary/ > /etc/apt/sources.list.d/jenkins.list'

Step 2: Install Jenkins

sudo apt update
sudo apt install jenkins

Step 3: Start Jenkins

sudo systemctl start jenkins

Step 4: Enable Jenkins to Start on the Boot

sudo systemctl enable jenkins

3. Install Jenkins on Windows

Step 1: Download Jenkins

  • Go to the Jenkins official website.

  • Download the Windows installer (.msi).

Step 2: Run the Installer

  • Double-click the .msi file and follow the installation wizard.

  • Jenkins will be installed as a Windows service.

Step 3: Start Jenkins

  • Open your browser and navigate to http://localhost:8080.

4. Install Jenkins in Docker

Step 1: Install Docker

Ensure Docker is installed on your system. You can verify with:

docker --version

Step 2: Pull the Jenkins Image

docker pull jenkins/jenkins:lts

Step 3: Run the Jenkins Container

docker run -d -p 8080:8080 -p 50000:50000 --name jenkins jenkins/jenkins:lts

5. Access Jenkins

Once Jenkins is running, access it via a web browser:

  • URL: http://<your_server_ip>:8080

    • On a local machine: http://localhost:8080.

6. Complete Initial Setup

Unlock Jenkins

  • When you access Jenkins for the first time, it will prompt for an Administrator Password.

  • The password can be found in the following file:

      cat /var/lib/jenkins/secrets/initialAdminPassword

    Or for Docker:

      docker exec -it jenkins cat /var/jenkins_home/secrets/initialAdminPassword

Install Plugins

  • Jenkins will suggest essential plugins. Install the recommended plugins, or customize your selection based on your needs.

Create an Admin User

  • Set up your administrator account.

7. Verify Installation

After completing the setup, you’ll be redirected to the Jenkins dashboard, where you can start creating jobs and configuring pipelines.

Pipeline Structure:

Creating a Freestyle Job in Jenkins to print "Hello World!!" is straightforward.

Step 1: Open Jenkins Dashboard

  1. Navigate to your Jenkins instance:
    URL: http://<your_server_ip>:8080
    For local setups: http://localhost:8080.

  2. Log in with your credentials.

Step 2: Create a Freestyle Project

  1. Click on "New Item" in the Jenkins dashboard.

  2. Enter the Name for the job, e.g., HelloWorldJob.

  3. Select Freestyle Project.

  4. Click OK.

Step 3: Configure the Job

  1. In the job configuration screen:

    • Add a description (optional), e.g., "A simple pipeline to print Hello World!".

  2. Scroll to the Build section:

    • Click "Add Build Step".

    • Select "Execute Shell" (on Linux/Mac).

  3. Enter the following command:

    • For Linux/Mac:

        echo "Hello World!!"

Step 4: Save and Run the Job

  1. Click Save at the bottom of the configuration screen.

  2. In the project dashboard, click Build Now to trigger the job.

Step 5: View the Output

  1. Go to the Build History section on the left side and click on the latest build (e.g., #1).

  2. Click "Console Output" to view the results.

  3. You should see:

     Started by user <your username>
 Building in workspace /var/lib/jenkins/workspace/HelloWorldJob
 [HelloWorldJob] $ /bin/sh -xe /tmp/jenkins123.sh
 + echo Hello World!!
 Hello World!!
 Finished: SUCCESS

Optional Enhancements

  • Add Parameters: Allow users to input custom messages.

  • Trigger Builds: Set triggers (e.g., periodic builds, Git commits).

  • Archive Artifacts: Save the output to a file if needed.

What is CI/CD?

What is CI/CD?

CI/CD stands for Continuous Integration (CI) and Continuous Delivery (or Deployment) (CD). It is a methodology in DevOps that automates and streamlines the software development lifecycle (SDLC) to improve quality, speed, and reliability in delivering applications to users.

Continuous Integration (CI)

CI is the practice of merging code changes from multiple developers into a shared repository frequently (often several times a day). Automated processes are triggered to build, test, and validate the code to ensure it integrates seamlessly.

Key Aspects of CI:

  1. Automated Builds:

    • Code changes are automatically compiled after being pushed to the repository.

  2. Automated Testing:

    • Unit, integration, and sometimes functional tests are automatically executed to identify bugs early.

  3. Feedback Loop:

    • Developers get immediate feedback on whether their code breaks the build or fails tests.

Benefits of CI:

  • Reduces integration issues by detecting errors early.

  • Encourages smaller, incremental code changes.

  • Speeds up the development process with automated testing.

Continuous Delivery (CD)

CD is the practice of automating the release of validated code to staging or production environments. It ensures that the code is always in a deployable state.

Key Aspects of CD:

  1. Automated Deployment Pipeline:

    • Code changes that pass CI are deployed to staging environments for further validation.

  2. Approval Gates:

    • CD pipelines may include manual approvals before pushing to production.

  3. Frequent Releases:

    • Encourages small, incremental updates to reduce the risk of big releases.

Benefits of CD:

  • Ensures a consistent and predictable release process.

  • Reduces deployment risks by enabling smaller and more frequent updates.

  • Makes it easier to roll back in case of issues.

Continuous Deployment (CD - Extended)

Continuous Deployment takes Continuous Delivery one step further by automatically deploying validated changes to production without manual intervention.

Difference Between Continuous Delivery and Continuous Deployment:

AspectContinuous DeliveryContinuous Deployment
Deployment to ProductionRequires manual approvalFully automated
Use CaseSuitable for environments with strict complianceIdeal for fast-moving applications

Why is CI/CD Important?

  1. Faster Time-to-Market:

    • Automates repetitive tasks, reducing the time it takes to develop, test, and release code.

  2. Higher Quality:

    • Automated tests ensure bugs are caught early and frequently, improving software quality.

  3. Reduced Risks:

    • Incremental updates are easier to test and deploy, reducing the risks of major failures.

  4. Improved Collaboration:

    • Promotes teamwork and reduces friction in merging code changes.

Example Workflow for CI/CD

  1. Developer Workflow:

    • A developer pushes code to a version control system (e.g., GitHub, GitLab).

  2. Continuous Integration:

    • Jenkins, GitLab CI, or another CI tool builds the code.

    • Automated tests run to ensure functionality.

  3. Continuous Delivery:

    • Validated builds are deployed to staging.

    • Additional testing and approvals take place.

  4. Continuous Deployment:

    • Changes are deployed to production (if automated deployment is enabled).

Common Tools Used in CI/CD

  • CI Tools: Jenkins, GitLab CI, CircleCI, Travis CI, GitHub Actions.

  • CD Tools: ArgoCD, Spinnaker, Octopus Deploy.

  • Version Control: Git (GitHub, GitLab, Bitbucket).

  • Containerization: Docker.

  • Orchestration: Kubernetes.

What Is a Build Job?

A build job in Jenkins is a task or a project that Jenkins executes to achieve a specific outcome, such as compiling source code, running tests, or deploying applications. It serves as the basic unit of work in Jenkins, representing a sequence of steps that automate tasks in your software development lifecycle (SDLC).

Key Characteristics of a Build Job

  1. Automation:

    • A build job automates manual tasks, such as compiling code, running tests, or deploying artifacts.

  2. Reproducibility:

    • Build jobs ensure that the process can be executed repeatedly with consistent results.

  3. Customization:

    • You can define what tasks a building job performs by configuring build steps, environment variables, and triggers.

  4. Flexibility:

    • Jenkins supports multiple types of jobs (e.g., Freestyle, Pipeline, Multibranch Pipeline) for different use cases.

Types of Build Jobs in Jenkins

  1. Freestyle Project:

    • A basic build job with configurable steps like compiling code or running shell scripts.

  2. Pipeline Job:

    • A job defined using code (Jenkinsfile), offers

    • more flexibility for complex CI/CD workflows.

  3. Multibranch Pipeline:

    • Automatically creates pipelines for each branch in a repository.

  4. Folder:

    • Groups related jobs together for better organization.

  5. External Job:

    • Monitors jobs that are run outside of Jenkins.

Common Tasks Performed by Build Jobs

  1. Compile Code:

    • Use tools like Maven, Gradle, or Ant to compile the source code.

  2. Run Tests:

    • Execute unit tests, integration tests, or functional tests automatically.

  3. Build Artifacts:

    • Package compiled code into deployable artifacts (e.g., .jar, .war, .zip files).

  4. Deploy Code:

    • Automate deployments to staging or production environments.

  5. Code Analysis:

    • Perform static code analysis using tools like SonarQube.

  6. Trigger Other Jobs:

    • A build job can trigger other dependent jobs in a pipeline.

How Does a Build Job Work?

  1. Source Code Checkout:

    • Jenkins pulls code from a repository (e.g., Git, SVN).

  2. Build Execution:

    • Executes the defined build steps (e.g., Maven build, shell commands).

  3. Testing:

    • Runs automated tests to validate the build.

  4. Post-Build Actions:

    • Executes actions like sending notifications, archiving artifacts, or triggering downstream jobs.

  5. Feedback:

    • Provides feedback (success/failure) via logs, emails, or dashboards.

Benefits of Using Build Jobs

  1. Automation:

    • Reduces manual effort and ensures consistency.

  2. Efficiency:

    • Speeds up the development lifecycle by automating tasks.

  3. Error Detection:

    • Detects bugs early through automated builds and tests.

  4. Collaboration:

    • Enables teams to share feedback and results quickly.

What is Freestyle Projects?

A Freestyle Project in Jenkins is a flexible and straightforward type of job that allows users to define custom build steps. It is the most basic type of Jenkins job, commonly used for simple build, test, and deployment workflows.

Key Features of Freestyle Projects

  1. Flexibility:

    • You can configure build steps, such as running shell commands, executing batch scripts, or calling external tools.

  2. Customizable:

    • Supports a variety of configurations, including build triggers, environment variables, and post-build actions.

  3. Wide Compatibility:

    • Works with many external tools and plugins for tasks like version control integration, notifications, and artifact archiving.

  4. Ease of Use:

    • Ideal for beginners due to its simple and intuitive configuration interface.

Use Cases for Freestyle Projects

  1. Building Source Code:

    • Compile code using tools like Maven, Gradle, or Ant.

  2. Running Automated Tests:

    • Execute unit, integration, or functional tests.

  3. Deploying Applications:

    • Automate deployment to staging or production environments.

  4. Executing Custom Scripts:

    • Run shell scripts, batch files, or other command-line tools.

  5. Triggering Other Jobs:

    • Freestyle jobs can trigger downstream builds or other projects.

How to Create a Freestyle Project

  1. Open Jenkins Dashboard:

  2. Create a New Job:

    • Click on "New Item".

    • Enter a name for the project, e.g., MyFreestyleProject.

    • Select Freestyle Project and click OK.

  3. Configure the Job:

    • Source Code Management:

      • Link to a Git repository or other version control systems.

    • Build Triggers:

      • Define triggers such as periodic builds, SCM polling, or webhook-based triggers.

    • Build Steps:

      • Add steps like:

        • Running shell scripts (echo "Hello World!").

        • Compiling code with Maven or Gradle.

    • Post-Build Actions:

      • Set actions like sending notifications, archiving artifacts, or triggering other jobs.

  4. Save and Run:

    • Click Save to finalize the configuration.

    • Trigger the job by clicking "Build Now".

Limitations of Freestyle Projects

  1. Lack of Pipeline as Code:

    • Freestyle Projects do not support defining workflows as code (like Jenkins Pipelines using Jenkinsfiles).

  2. Complexity in Scaling:

    • Managing dependencies and stages can become cumbersome for complex CI/CD workflows.

  3. Less Modern:

    • Freestyle jobs are considered less suitable for modern DevOps workflows compared to pipelines.

When to Use a Freestyle Project

  • For straightforward tasks like building, testing, or deploying small projects.

  • For jobs that don’t require complex branching, multiple stages, or dependencies.

  • For prototyping or experimenting with Jenkins features.

Task-01

  • create an agent for your app. ( which you deployed from docker in the earlier task)

  • Create a new Jenkins freestyle project for your app.

  • In the "Build" section of the project, add a build step to run the "docker build" command to build the image for the container.

  • Add a second step to run the "docker run" command to start a container using the image created in step 3.

To accomplish Task-01, here's a detailed step-by-step guide to creating a Jenkins Freestyle Project that automates the build and run of a Dockerized app:

Prerequisites

  1. Ensure your Jenkins instance has access to the Docker CLI.

    • Jenkins should be installed on a system with Docker installed, or Docker must be accessible from Jenkins via a Docker socket.

    • The Jenkins user should have the required permissions to run Docker commands.

  2. The application code (with a Dockerfile) should be in a Git repository.

  3. Verify the docker command works by running:

     docker --version

Step-by-Step Guide

Step 1: Create a Jenkins Freestyle Project

  1. Log in to the Jenkins dashboard (http://<your_server_ip>:8080).

  2. Click "New Item" on the dashboard.

  3. Enter a name for your project, e.g., DockerizedAppProject.

  4. Select Freestyle Project and click OK.

Step 2: Configure the Freestyle Project

a. Source Code Management

  1. In the project configuration, go to the Source Code Management section.

  2. Choose Git and provide the repository URL where your application code and Dockerfile are hosted (e.g., GitHub or GitLab repo).

  3. Add credentials if required.

b. Build Triggers (Optional)

  1. Set up build triggers if needed:

    • Poll SCM: Automatically triggers builds on code changes.

    • Webhook: Triggers build when changes are pushed.

Step 3: Add Build Steps

a. Step 1: Build a Docker Image

  1. In the Build section, click Add Build Step and select Execute Shell.

  2. Add the command to build the Docker image:

     docker build -t myapp:latest .

    • myapp:latest is the name and tag for your Docker image.

    • Ensure the Dockerfile is in the root directory of your repository.

b. Step 2: Run Docker Container

  1. Add another Execute Shell build step.

  2. Add the command to run a Docker container:

     docker run -d -p 8080:8080 --name myapp_container myapp:latest

    • -d: Runs the container in detached mode.

    • -p 8080:8080: Maps port 8080 of the container to port 8080 of the host.

    • --name myapp_container: Assign a name to the container.

Step 4: Save and Build

  1. Click Save to apply the configuration.

  2. On the project dashboard, click Build Now to trigger the build.

  3. Monitor the Build History and view the Console Output to ensure the commands execute successfully.

Verification

  1. Check the Docker Image:

    • Run:

        docker images

    • Verify that the myapp:latest image is listed.

  2. Check the Running Container:

    • Run:

        docker ps

    • Confirm the myapp_container is running and mapped to port 8080.

  3. Access the Application:

    • Open a web browser and navigate to http://<your_server_ip>:8080.

Post-Build Cleanup (Optional)

To prevent the accumulation of unused Docker images and containers:

  1. Add a Post-Build Action or periodic cleanup script:

     docker rm -f myapp_container || true
 docker rmi myapp:latest || true

Task-02

  • Create Jenkins project to run the "docker-compose up -d" command to start the multiple containers defined in the compose file (Hint- use day-19 Application & Database docker-compose file)

  • Set up a cleanup step in the Jenkins project to run the "docker-compose down" command to stop and remove the containers defined in the compose file.

Here's how you can accomplish Task-02 to create a Jenkins project that runs docker-compose up -d and includes a cleanup step with docker-compose down.

Prerequisites

  1. Ensure the Docker Compose file (docker-compose.yml) is prepared and tested.

    • The file should define the application and database containers as needed (referencing Day 19).

  2. Verify that Docker and Docker Compose are installed on the Jenkins server.

    • Run the following commands to confirm:

        docker --version
  docker-compose --version

  3. Jenkins must have sufficient permissions to execute Docker Compose commands.

Step-by-Step Guide

Step 1: Create a New Jenkins Freestyle Project

  1. Log in to the Jenkins dashboard (http://<your_server_ip>:8080).

  2. Click "New Item".

  3. Enter a project name, e.g., DockerComposeAppProject.

  4. Select Freestyle Project and click OK.

Step 2: Configure the Freestyle Project

a. Source Code Management

  1. In the Source Code Management section:

    • Select Git.

    • Enter the repository URL where your docker-compose.yml file is stored.

    • Add credentials if necessary.

b. Build Triggers (Optional)

  1. Configure build triggers as needed:

    • Poll SCM: Automatically trigger builds when changes are detected in the repository.

    • Webhook: Trigger builds when changes are pushed.

Step 3: Add Build Steps

a. Step 1: Start Containers Using Docker Compose

  1. Scroll to the Build section and click Add Build Step → Execute Shell.

  2. Add the command to bring up the containers:

     docker-compose up -d
    • This command will start all containers defined in the docker-compose.yml file in detached mode.

Step 4: Set Up Cleanup Step

a. Post-Build Cleanup

  1. Scroll to the Post-Build Actions section (or add a step in Build).

  2. Add another Execute Shell step and enter the command:

     docker-compose down
    • This will stop and remove all containers, networks, and volumes defined in the docker-compose.yml file.

Step 5: Save and Run the Job

  1. Click Save to apply the configuration.

  2. On the project dashboard, click Build Now to trigger the project.

Verification

After docker-compose up -d Step

  1. Check the running containers:

     docker ps
    • You should see the containers for your application and database running.

  2. Verify the application:

    • Access the application via the browser or API endpoint (http://<your_server_ip>:<app_port>).

After docker-compose down Step

  1. Check for stopped containers:

     docker ps -a
    • Confirm that no containers from the project are listed.

  2. Check for removed networks and volumes if applicable:

     docker network ls
 docker volume ls

Tips and Enhancements

  1. Environment Variables:

    • Use a .env file for dynamic configuration in docker-compose.yml.

  2. Artifact Management:

    • Archive logs or other artifacts generated during container runtime for debugging purposes.

  3. Error Handling:

    • Use || true to gracefully handle failures in docker-compose down (e.g., if no containers are running).

  4. Job Chaining:

    • Use Jenkins pipelines to chain this job with other tasks like automated testing or deployments.
JenkinsPipelinesCI/CDbuild job#freestyle-jenkins-project