Jenkins can run stages in parallel, and most people only ever use it serially.
Let’s see what that looks like. Imagine you have a Jenkinsfile like this:
pipeline {
agent any
stages {
stage('Build') {
steps {
echo 'Building...'
sleep 5 // Simulate build time
}
}
stage('Test') {
steps {
echo 'Testing...'
sleep 5 // Simulate test time
}
}
stage('Deploy') {
steps {
echo 'Deploying...'
sleep 5 // Simulate deploy time
}
}
}
}
This pipeline runs Build, then Test, then Deploy. Each stage takes 5 seconds. The total time is 15 seconds.
Now, what if Test and Deploy don’t actually depend on each other? We can run them at the same time. Here’s how you’d modify the Jenkinsfile:
pipeline {
agent any
stages {
stage('Build') {
steps {
echo 'Building...'
sleep 5
}
}
stage('Run Tests and Deploy') {
parallel {
stage('Test') {
steps {
echo 'Testing...'
sleep 5
}
}
stage('Deploy') {
steps {
echo 'Deploying...'
sleep 5
}
}
}
}
}
}
With this change, the Build stage still runs first. But once Build is complete, Jenkins kicks off both the Test and Deploy stages simultaneously. Since they both take 5 seconds, and they run in parallel, the Run Tests and Deploy stage (the container for the parallel stages) will complete in about 5 seconds. The total pipeline time drops from 15 seconds to 10 seconds.
The real power comes when you have more stages that can be independent. Consider a scenario where you need to run unit tests, integration tests, and a security scan after your build, and none of these absolutely require another to finish first.
pipeline {
agent any
stages {
stage('Build') {
steps {
echo 'Building...'
sleep 5
}
}
stage('Post-Build Checks') {
parallel {
stage('Unit Tests') {
steps {
echo 'Running Unit Tests...'
sleep 7
}
}
stage('Integration Tests') {
steps {
echo 'Running Integration Tests...'
sleep 10
}
}
stage('Security Scan') {
steps {
echo 'Running Security Scan...'
sleep 5
}
}
}
}
stage('Final Deploy') {
steps {
echo 'Deploying...'
sleep 3
}
}
}
}
In this example, Build takes 5 seconds. Then, Unit Tests (7s), Integration Tests (10s), and Security Scan (5s) all start at the same time. The longest of these parallel stages dictates the duration of the Post-Build Checks stage, which is 10 seconds (from Integration Tests). Finally, Final Deploy takes 3 seconds. The total pipeline time is 5s (Build) + 10s (longest parallel stage) + 3s (Final Deploy) = 18 seconds.
Without parallel execution, this pipeline would take 5s + 7s + 10s + 5s + 3s = 30 seconds. That’s a 40% reduction in total runtime just by identifying and parallelizing independent stages.
The parallel block in Jenkinsfile is quite flexible. You can nest parallel blocks within other parallel blocks, or have a parallel stage within a larger serial stage, or vice-versa. The key is to analyze your workflow and identify tasks that don’t have strict sequential dependencies. For instance, if you have multiple distinct test suites that can run independently, or if you need to deploy to different environments that don’t rely on each other, those are prime candidates for parallelization.
To get the most out of parallel stages, you’ll want to ensure your Jenkins agents have sufficient capacity. If you have 10 stages that can run in parallel but only 2 available executors on your Jenkins controller, they’ll still run sequentially in groups of 2. You can configure the number of executors per agent in Manage Jenkins > Nodes > [Your Node Name] > Configure > Number of executors. It’s also crucial that your build jobs themselves are stateless or can be made to run concurrently without side effects. If two parallel jobs try to modify the same file or lock the same resource, you’ll run into race conditions and unexpected failures.
The trick to truly unlocking the power of parallel execution is understanding that a stage block is just a logical grouping of steps. When you put multiple stage blocks inside a parallel block, Jenkins treats each of those inner stage blocks as an independent unit of work that can be scheduled concurrently. The parallel block itself will only complete once all of its contained stages have finished. This is why the longest-running stage within a parallel block determines the duration for that entire parallel section.
When you’re debugging a pipeline that uses parallel stages, it’s easy to get lost in the output. The Jenkins UI renders parallel stages as branches in a tree-like structure. Clicking on a specific stage in the build history view will show you the logs and status for that individual stage. This is essential for pinpointing which parallel task is taking too long or failing.
If you have a stage that needs to be executed only after a specific set of parallel stages have completed, you can define a new stage that depends on the completion of the parallel stage group. This is achieved by structuring your pipeline to have a serial stage that contains the parallel stages, and then another serial stage that follows it. The second serial stage will only begin execution after the entire parallel block it logically follows has finished.
The ability to run stages in parallel is a fundamental optimization technique in CI/CD, and it’s often overlooked by teams who default to linear pipeline definitions. By carefully analyzing your build and deployment workflows for independent tasks, you can significantly reduce your pipeline execution times, leading to faster feedback loops for developers and quicker delivery of features.
The next step after optimizing individual pipelines is to consider distributed builds across multiple Jenkins controllers or cloud-based build farms, which introduces its own set of challenges around state management and artifact sharing.