When you spin up test environments with Docker Compose in Jenkins, the most surprising truth is that the ephemeral nature of these environments is their greatest strength, not a weakness to be fought against.
Let’s see this in action. Imagine a Jenkins pipeline that needs to test a web service against a database.
# Jenkinsfile
pipeline {
agent any
stages {
stage('Setup Test Environment') {
steps {
script {
// Define the Docker Compose configuration inline for simplicity
// In a real scenario, this would be a separate docker-compose.yml file
dockerCompose = """
version: '3.8'
services:
webapp:
image: nginx:latest
ports:
- "8080:80"
depends_on:
- database
database:
image: postgres:13
environment:
POSTGRES_DB: testdb
POSTGRES_USER: testuser
POSTGRES_PASSWORD: password123
volumes:
- db_data:/var/lib/postgresql/data
volumes:
db_data:
"""
// Write the compose file to disk
writeFile file: 'docker-compose.yml', text: dockerCompose
sh 'docker compose up -d'
}
}
}
stage('Run Tests') {
steps {
script {
// In a real test stage, you'd run your test suite here.
// For demonstration, we'll just check if the services are up.
sh 'docker compose ps'
// Example: curl http://localhost:8080 to test the webapp
// sh 'curl http://localhost:8080'
}
}
}
stage('Teardown Test Environment') {
steps {
script {
sh 'docker compose down -v' // -v to remove volumes
}
}
}
}
}
This Jenkinsfile defines three stages: setting up the environment, running tests, and tearing it down. The Setup Test Environment stage writes a docker-compose.yml file directly into the workspace and then uses docker compose up -d to start the services defined in it in detached mode. The Run Tests stage would contain your actual test execution logic, interacting with the services started in the previous stage. Finally, Teardown Test Environment uses docker compose down -v to stop and remove all containers, networks, and importantly, the volumes associated with the test environment.
The problem this solves is the classic "it works on my machine" syndrome, amplified in CI/CD. By defining the entire test environment—databases, caches, other services—as code in a docker-compose.yml file, you ensure that the environment running your tests is identical to the environment defined. Jenkins, with its Docker integration, can execute these compose commands, spinning up precisely what’s needed for a given build.
Internally, docker compose up reads the docker-compose.yml file, translates its services into Docker container definitions, and orchestrates their creation and networking. Each service becomes an independent container, with dependencies managed by Docker Compose. depends_on in the compose file ensures that the webapp container only starts after the database container is healthy (or at least running, depending on the compose version and configuration). The ports directive maps a host port (8080) to the container’s port (80), making the web service accessible from the Jenkins agent. The volumes directive for the database ensures that data persists if needed between container restarts, though for ephemeral test environments, this is often used to provide initial data or to clean up entirely.
When you run docker compose down -v, Docker Compose sends a stop signal to all containers defined in the compose file, waits for them to exit, and then removes them. The -v flag is crucial for true ephemeral environments as it instructs Docker to also remove any anonymous volumes defined in the compose file, ensuring a clean slate for the next test run. This prevents leftover data from corrupting future tests or consuming disk space.
Most people understand that docker compose down removes containers. What’s less obvious is that the volumes defined within the docker-compose.yml file itself (anonymous volumes) are not removed by default with docker compose down. You must explicitly use the -v flag to get rid of them, otherwise, your disk will fill up with old database data, cache data, or whatever else your services wrote to volumes.
The next concept to explore is how to manage more complex multi-host Docker Compose setups or integrate Docker Swarm for more sophisticated orchestration needs.