Jenkins can get pretty finicky about what state it carries between pipeline stages. If you’re trying to pass build artifacts from one stage to the next, you’ll quickly run into a wall if you don’t explicitly tell Jenkins what to save.

Imagine you’re building a Docker image in the first stage and want to push it in the second. You can’t just assume the docker build output from stage 1 is magically available in stage 2. Jenkins, by default, treats each stage as a relatively isolated execution environment.

Here’s a typical scenario:

pipeline {
    agent any
    stages {
        stage('Build') {
            steps {
                script {
                    // Let's simulate creating a file
                    sh 'echo "This is my build output" > build_output.txt'
                    sh 'docker build -t my-app:latest .'
                }
            }
        }
        stage('Deploy') {
            steps {
                script {
                    // This will fail because build_output.txt is not here!
                    sh 'cat build_output.txt'
                    // And you can't directly access the docker image created in the previous stage
                    sh 'docker push my-app:latest'
                }
            }
        }
    }
}

The cat build_output.txt command in the 'Deploy' stage will fail with a "No such file or directory" error. The Docker image my-app:latest built in the 'Build' stage is also not available for pushing in the 'Deploy' stage.

The fundamental problem is how Jenkins manages workspace state between stages. By default, a new workspace is often checked out or reset for each stage, or at least the intermediate files are not explicitly preserved. You need a mechanism to tell Jenkins, "Hey, save this specific file or these specific files from the previous stage so I can use them here."

The solution is Jenkins’ built-in archiveArtifacts step and the stash/unstash steps.

archiveArtifacts: For Permanent Storage

If you want to keep artifacts after the build is complete, archiveArtifacts is your go-to. This makes them accessible via the Jenkins UI for download.

Diagnosis: You’re trying to access a file in a later stage, and it’s missing. The build log might show file not found errors.

Cause 1: Artifacts not explicitly archived.

  • Diagnosis: Check your Jenkinsfile. Do you see an archiveArtifacts step in the stage before the one where you need the files?
  • Fix: Add archiveArtifacts to the end of the stage that produces the files. For example, to save build_output.txt:
    stage('Build') {
        steps {
            script {
                sh 'echo "This is my build output" > build_output.txt'
                // Archive the file
                archiveArtifacts artifacts: 'build_output.txt', fingerprint: true
            }
        }
    }
    
    The fingerprint: true is good practice; it helps track where artifacts come from and go to across builds.
  • Why it works: archiveArtifacts tells Jenkins to copy the specified files from the workspace to a persistent storage location associated with the build. These files are then available from the build’s artifact section in the UI. This doesn’t directly make them available in the workspace of a subsequent stage, but it’s the primary way to keep them.

Cause 2: Incorrect archiveArtifacts pattern.

  • Diagnosis: You’ve used archiveArtifacts, but the files aren’t showing up, or you’re missing specific ones.
  • Fix: Ensure your glob patterns are correct. If you have files in subdirectories, you need to account for that. To archive everything in a dist directory:
    archiveArtifacts artifacts: 'dist/**/*', fingerprint: true
    
    To archive specific files across multiple directories:
    archiveArtifacts artifacts: 'logs/*.log, reports/*.html', fingerprint: true
    
  • Why it works: The pattern matching for archiveArtifacts uses standard shell globbing. An incorrect pattern simply won’t select the files you intend.

stash/unstash: For Passing Between Stages

If you need files within the pipeline execution itself, to be used in a subsequent stage’s workspace, stash and unstash are the tools. stash saves files to a temporary area managed by Jenkins, and unstash retrieves them into the current stage’s workspace.

Diagnosis: You’re trying to cat or docker push a file/image that was created in a previous stage, and it’s missing from the current stage’s workspace.

Cause 3: stash not used in the producing stage.

  • Diagnosis: The files you need are present in the workspace of the first stage, but not in the workspace of the second stage.
  • Fix: Add a stash step in the stage before the one that needs the files.
    stage('Build') {
        steps {
            script {
                sh 'echo "This is my build output" > build_output.txt'
                sh 'docker build -t my-app:latest .'
                // Stash the file and the Docker image context if needed for later
                stash includes: 'build_output.txt', name: 'build_data'
                // For Docker images, you typically stash the Dockerfile and context if needed,
                // or push the image and pull it in the next stage.
                // If you need the image itself, it's more complex and often involves
                // saving the image tarball or using a registry.
                // For simplicity here, let's assume we're passing a file for demonstration.
            }
        }
    }
    
    The name parameter is crucial; it’s how you’ll refer to this stashed bundle later.
  • Why it works: stash takes files from the current workspace and uploads them to Jenkins’ internal storage, associated with the build. It doesn’t put them back into the workspace immediately, but it makes them available for unstash.

Cause 4: unstash not used in the consuming stage.

  • Diagnosis: You’ve stashed files, but they are still not found in the workspace of the next stage.
  • Fix: Add an unstash step in the stage that requires the files, using the same name you used in stash.
    stage('Deploy') {
        steps {
            script {
                // Unstash the files
                unstash 'build_data'
                // Now build_output.txt should be available
                sh 'cat build_output.txt'
                // If you had stashed Dockerfile and context, you'd build again here.
                // For the actual image, see Cause 6.
            }
        }
    }
    
  • Why it works: unstash retrieves the files previously saved by stash and places them into the current stage’s workspace, making them directly accessible by subsequent steps in that stage.

Cause 5: Incorrect stash/unstash includes pattern.

  • Diagnosis: You’ve used stash and unstash, but specific files are still missing or incorrect files are being transferred.
  • Fix: Ensure the includes pattern in stash correctly targets the files you want. The unstash step doesn’t take an includes pattern; it retrieves everything stashed under the given name.
    // In the producing stage
    stash includes: 'build_output.txt', name: 'build_data'
    stash includes: 'config/*.yml', name: 'config_files'
    
    // In the consuming stage
    unstash 'build_data'
    unstash 'config_files'
    
  • Why it works: Like archiveArtifacts, the includes pattern in stash uses globbing. If it’s too broad, you might stash unwanted files; if it’s too narrow, you’ll miss files.

Cause 6: Stashing Docker images or large build artifacts.

  • Diagnosis: You’re trying to stash a Docker image or a very large set of build artifacts, and it’s slow, fails, or consumes excessive Jenkins disk space.
  • Fix: For Docker images, the standard practice is to push to a Docker registry (like Docker Hub, ECR, GCR) in the build stage and pull in the deploy stage.
    stage('Build') {
        steps {
            script {
                sh 'docker build -t my-registry/my-app:${env.BUILD_ID} .'
                sh 'docker push my-registry/my-app:${env.BUILD_ID}'
            }
        }
    }
    stage('Deploy') {
        steps {
            script {
                sh 'docker pull my-registry/my-app:${env.BUILD_ID}'
                // Now you can use the pulled image
            }
        }
    }
    
    For large build artifacts that don’t need to be in a registry, consider archiving them (archiveArtifacts) and downloading them in the next stage if truly necessary, or restructuring your pipeline to avoid needing them directly in the workspace.
  • Why it works: stash is designed for transferring files between pipeline stages, not for managing large, persistent build outputs like Docker images. Registries are optimized for storing and distributing container images.

After correctly stashing and unstashing build_output.txt, the cat build_output.txt command in the 'Deploy' stage will succeed. If you were managing Docker images via a registry, the push/pull logic would also work.

The next error you’ll likely encounter is related to managing build environments or credentials for deployment.

Want structured learning?

Take the full Jenkins course →