Jenkins’ Git plugin can be a real bottleneck, especially on large repositories or slow network connections. The culprit is often the full clone operation it performs by default, downloading the entire history of your repository on every checkout. This article explains how to dramatically speed this up using shallow cloning.
Here’s a Jenkins job configured to use a shallow clone:
pipeline {
agent any
stages {
stage('Checkout') {
steps {
checkout([
$class: 'GitSCM',
branches: [[name: '*/main']],
userRemoteConfigs: [[url: 'git@github.com:your-org/your-repo.git']],
extensions: [
[$class: 'CloneOption', depth: 1, shallow: true]
]
])
}
}
stage('Build') {
steps {
// Your build steps here
sh 'echo "Building..."'
}
}
}
}
The magic happens in the extensions block, specifically the CloneOption with depth: 1 and shallow: true. This tells the Git plugin to perform a shallow clone, fetching only the most recent commit and a specified number of preceding commits (in this case, just the latest one).
The fundamental problem this solves is the sheer volume of data transferred during a git clone or git fetch operation. When Jenkins checks out a branch for a build, it often needs to fetch the latest changes. Without shallow cloning, it performs a full clone, which involves downloading the entire commit history, all branches, and all tags. For repositories with years of history and hundreds or thousands of commits, this can amount to gigabytes of data, taking minutes or even hours to download. This unnecessary download directly translates to longer build times and increased network traffic.
When you enable shallow cloning with depth: 1, Git instructs the remote server to only send back the most recent commit object and its parent(s), up to the specified depth. It doesn’t download any historical commit data beyond that. This drastically reduces the amount of data that needs to be transferred over the network. For the build itself, only the latest code snapshot is required, making the full history irrelevant for most CI/CD use cases.
The depth parameter in CloneOption controls how many commits are fetched. depth: 1 means only the very latest commit is downloaded. If your build process requires a slightly deeper history (e.g., to analyze commit messages for specific changes within the last few commits), you could increase this value. However, for most typical CI builds, depth: 1 is sufficient and offers the most significant speedup.
The shallow: true parameter is what actually activates the shallow clone behavior. Without it, even specifying a depth might not result in a truly shallow clone, as Git’s behavior can vary. Explicitly setting shallow: true ensures the plugin uses the --depth and --shallow-since options correctly when interacting with the Git remote.
If you’re using the Jenkins Git plugin with a Pipeline job, you can configure shallow cloning directly within your Jenkinsfile as shown above. For freestyle projects, you’d find this option under the "Source Code Management" section, within the Git configuration, under "Additional Behaviours" -> "Advanced clone behaviours", where you can check "Shallow clone" and set the depth.
The primary benefit is a dramatic reduction in checkout time. On large repositories, this can shrink checkout times from minutes to seconds. This directly contributes to faster feedback loops for developers, as builds complete more quickly. Furthermore, it reduces the load on your Git server and your Jenkins build agents by minimizing network I/O and disk space usage for the .git directory.
There’s a subtle but important consequence of shallow clones: Git’s history is incomplete. If you try to perform operations that rely on history beyond the shallow depth, such as git log -n 100 (where 100 is more than your depth) or git blame on older lines, Git will report that it cannot find the necessary objects and will prompt you to fetch more history. For typical CI/CD workflows that only need the latest code, this is rarely an issue, but it’s a crucial detail to understand if you ever need to perform deeper historical analysis within the build job itself.
When Jenkins needs to update an existing workspace with a shallow clone, it typically performs a git fetch --depth=<depth> followed by a git reset --hard. This is efficient because it only fetches the new commits required to reach the desired depth and then updates the workspace to point to the latest commit. This is much faster than a full git pull or git fetch --all followed by a git reset --hard.
Once your shallow clone is working, the next thing you might notice is that certain Git commands within your build scripts that expect full history will start failing.