GitLab runners can execute your CI/CD jobs, and setting them up with Docker and the shell executor is a common and flexible approach.
Here’s how to get a GitLab self-hosted runner configured to use Docker for job execution, leveraging the shell executor for the runner itself.
The Core Idea: Runner as a Dispatcher, Docker as the Worker
Think of the GitLab runner as a lightweight agent that polls GitLab for jobs. When it picks up a job, it then uses Docker to create a fresh, isolated environment to run that job’s commands. The "shell executor" part means the runner process itself runs directly on your host machine’s operating system, not inside a Docker container. It’s the intermediary that orchestrates Docker.
Step 1: Install the GitLab Runner Binary
First, you need the GitLab Runner executable on the machine that will host your runners. This machine needs to have Docker installed and running.
On a Linux system (like Ubuntu/Debian):
curl -L "https://packages.gitlab.com/install/repositories/runner/gitlab-runner/script.deb.sh" | bash
apt-get update
apt-get install gitlab-runner
This installs the latest gitlab-runner package and sets up a systemd service to manage it.
Step 2: Register Your Runner with GitLab
Now, you need to tell your GitLab instance about this new runner. You’ll need your GitLab instance’s URL and a registration token. You can find the registration token under your GitLab project’s Settings > CI/CD > Runners or for an instance-wide runner under Admin Area > CI/CD > Runners.
Run this command on your runner host:
sudo gitlab-runner register
You’ll be prompted for:
- GitLab instance URL:
https://your.gitlab.instance.com - Registration token:
your_registration_token - Description:
My Docker Shell Runner(or something descriptive) - Tags:
docker, shell, myproject(these are crucial for selecting which jobs this runner will pick up) - Executor:
docker(this tells the runner to use Docker for job execution) - Default Docker image:
ruby:2.7(this is the image used if a job doesn’t specify one. Choose a common one for your projects.)
After registration, the runner’s configuration will be saved in /etc/gitlab-runner/config.toml.
Step 3: Configure the Runner’s config.toml for Docker
The config.toml file is where you fine-tune how the runner interacts with Docker. Open it with sudo nano /etc/gitlab-runner/config.toml.
Here’s a typical setup for a Docker-enabled shell runner:
concurrent = 4
check_interval = 10
[[runners]]
name = "My Docker Shell Runner"
url = "https://your.gitlab.instance.com/"
token = "your_registration_token" # This is automatically managed, don't edit manually
executor = "docker"
limit = 2 # Max jobs this runner can run concurrently
environment = []
[runners.docker]
tls_cert_path = ""
image = "alpine:latest" # Default image if not specified in .gitlab-ci.yml
privileged = false
disable_cache = false
volumes = ["/cache"]
shm_size = 0
network_mode = "bridge" # Or host, or specific network
concurrent: The total number of jobs that can run on this runner host across all registered runners.check_interval: How often (in seconds) the runner polls GitLab for new jobs.limit(under[[runners]]): The maximum number of jobs this specific runner can execute concurrently.[runners.docker]: This section configures the Docker executor.image: The default Docker image to use if a job in your.gitlab-ci.ymldoesn’t specify one.alpine:latestis a common lightweight choice.privileged = false: For security, it’s best to keep thisfalse. Only enable if absolutely necessary and you understand the risks.volumes: This defines Docker volumes.["/cache"]is standard for GitLab CI’s cache mechanism.network_mode: Controls how the Docker containers connect to the network.bridgeis the default.hostgives containers direct access to the host’s network interfaces, which can be useful but less isolated.
Step 4: Start and Enable the Runner Service
Once configured, ensure the runner service is running and set to start on boot:
sudo gitlab-runner start
sudo gitlab-runner enable
You can check its status with:
sudo gitlab-runner status
Step 5: Configure Your .gitlab-ci.yml
Now, in your project’s .gitlab-ci.yml file, you need to ensure your jobs are tagged to be picked up by this runner and that they specify Docker images.
stages:
- build
- test
build_job:
stage: build
tags:
- docker # Must match a tag assigned during registration
- shell # Or another tag you used
image: node:16 # Specify a Docker image for this job
script:
- echo "Building the project..."
- npm install
- npm run build
test_job:
stage: test
tags:
- docker
- shell
image: node:16
script:
- echo "Running tests..."
- npm test
When a job is triggered, the runner will:
- Read the job’s
tagsand match them against its own registered tags. - If matched, it will pull the specified
image(node:16in this case) from Docker Hub (or your configured registry). - It will then start a Docker container from that image.
- The
scriptcommands will be executed inside that Docker container. - The container will be removed once the job is complete.
The Surprise: Shared Docker Volumes and Caching
A key aspect of config.toml is its volume configuration. When you define volumes = ["/cache"] within [runners.docker], you’re telling the runner to mount a host directory (usually managed by Docker itself, often within /var/lib/docker/volumes/gitlab-runner/_data/cache or similar) into the container at /cache. This is the mechanism GitLab CI uses for its caching. Files placed in /cache by one job will be available in /cache for subsequent jobs on the same runner, provided they use the same Docker volume and the cache isn’t invalidated. This is how build artifacts or downloaded dependencies can persist between stages or even between different pipeline runs.
The next step is to explore how to manage multiple runners, different executors, or advanced Docker configurations like custom Docker networks.