Jenkins is failing to write to its temporary directory because the underlying filesystem is full.

Here’s why that’s a problem: Jenkins uses temporary files for a lot of its operations, like unpacking archives, processing build logs, and managing workspaces. When the filesystem hosting these temporary files runs out of space, these operations fail abruptly, leading to build failures and a cascade of errors.

Common Causes and Fixes:

  1. Build Artifacts and Logs Accumulating: Jenkins can retain old build artifacts and logs indefinitely if not configured otherwise. Over time, these can consume significant disk space.

    • Diagnosis: Check the size of your Jenkins home directory, specifically JENKINS_HOME/jobs/<job_name>/builds/ and JENKINS_HOME/userContent/.
    • Fix: Implement a build retention strategy. In Jenkins, go to Manage Jenkins -> Configure System -> Global properties -> Build discarder. Select "Discard old builds" and choose a strategy like "Days to keep builds" (e.g., 7) or "Number of builds to keep" (e.g., 10). This will automatically clean up old build data.
    • Why it works: This policy tells Jenkins to periodically scan and remove build directories that exceed the specified retention criteria, freeing up disk space.
  2. Large Workspace Directories: If builds fail mid-execution, or if jobs are configured to not clean up their workspaces, these directories can grow enormous with downloaded dependencies, unpacked archives, and intermediate files.

    • Diagnosis: Identify large workspace directories. These are typically found in JENKINS_HOME/jobs/<job_name>/workspace/. Use du -sh JENKINS_HOME/jobs/<job_name>/workspace/ to check individual job workspace sizes.
    • Fix: Configure jobs to clean their workspaces before or after each build. In the job configuration, under "Build Environment," check "Delete workspace before build starts" or under "Post-build Actions," add a "Delete workspace when build is older than" option (e.g., 7 days).
    • Why it works: This ensures that stale or unnecessary files from previous builds are removed, preventing them from accumulating and consuming disk space.
  3. Temporary Directory (/tmp) Overflow: The system’s temporary directory, often /tmp, is used by Jenkins and other processes. If it fills up, Jenkins will have nowhere to put its own temporary files.

    • Diagnosis: Check the free space on the partition hosting /tmp. Run df -h /tmp. If it’s near 100% usage, that’s your problem.
    • Fix:
      • Option A (Clean up /tmp): Manually remove old, unneeded files from /tmp. Be cautious, as other running processes might be using these files. A safer approach is to reboot the server, which often clears /tmp.
      • Option B (Configure Jenkins to use a different temp dir): Set the java.io.tmpdir system property for Jenkins. This can be done by adding -Djava.io.tmpdir=/path/to/new/temp to the JAVA_ARGS in your Jenkins configuration file (e.g., /etc/sysconfig/jenkins or JENKINS_HOME/jenkins.xml). Ensure /path/to/new/temp exists and has ample space.
    • Why it works: Either frees up space in the default temporary directory or redirects Jenkins’ temporary file creation to a partition with more available space.
  4. Docker Volume Bloat (if Jenkins uses Docker): If your Jenkins agents or builds run within Docker containers, the Docker storage itself can become a bottleneck. This includes images, volumes, and build caches.

    • Diagnosis: Use docker system df to see Docker’s disk usage. Look for large volumes or dangling images.
    • Fix: Prune unused Docker resources. Run docker system prune -a --volumes. This removes all stopped containers, all networks not used by at least one container, all dangling images, and all dangling build cache. The --volumes flag is crucial for removing unused volumes.
    • Why it works: This cleans up Docker’s internal storage, reclaiming space used by old images, containers, and unattached volumes that are no longer needed.
  5. System Logs Filling Up: Large or verbose system logs, especially those generated by Jenkins itself or its plugins, can consume disk space on the partition where Jenkins is installed or where logs are directed.

    • Diagnosis: Check log file sizes. Common locations include /var/log/jenkins/jenkins.log or within JENKINS_HOME. Use du -sh /var/log/jenkins/ or du -sh JENKINS_HOME/logs/.
    • Fix: Configure log rotation for Jenkins logs. Edit /etc/logrotate.d/jenkins (or create it if it doesn’t exist) with a configuration like:
      /var/log/jenkins/jenkins.log {
          daily
          rotate 7
          compress
          missingok
          notifempty
          create 0640 jenkins adm
      }
      
      Then, run logrotate -f /etc/logrotate.d/jenkins to apply immediately.
    • Why it works: Log rotation ensures that log files are periodically compressed and old ones are deleted, preventing them from growing indefinitely.
  6. Large Plugin Data or Caches: Some Jenkins plugins store significant amounts of data or cache files within the JENKINS_HOME directory.

    • Diagnosis: Use du -sh JENKINS_HOME/* to identify large directories within your Jenkins home. Look for directories associated with specific plugins (e.g., JENKINS_HOME/plugins/<plugin_name>/, JENKINS_HOME/caches/).
    • Fix: Investigate the specific plugin. Some plugins have their own cleanup mechanisms or configuration options to manage their data size. For example, the "Artifact Manager" plugins might need configuration for artifact cleanup, or build scan plugins might need their cache limits adjusted.
    • Why it works: By managing or cleaning the data generated by specific plugins, you reduce the overall disk footprint of the Jenkins installation.

After resolving the disk space issue, you will likely encounter java.io.IOException: No space left on device errors related to Jenkins’ temporary directory.

Want structured learning?

Take the full Jenkins course →