GitLab CI pipelines can feel like a black box, but they’re actually a series of discrete jobs executed by runners, orchestrated by a .gitlab-ci.yml file.
Let’s build a pipeline for a typical microservice. Imagine a Python microservice with unit tests and a Docker image.
stages:
- build
- test
- deploy
variables:
IMAGE_NAME: registry.gitlab.com/$CI_PROJECT_PATH/my-microservice:$CI_COMMIT_SHORT_SHA
DOCKER_REGISTRY: registry.gitlab.com
docker-build:
stage: build
image: docker:20.10.16
services:
- docker:20.10.16-dind
script:
- echo "Building Docker image..."
- docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $DOCKER_REGISTRY
- docker build -t $IMAGE_NAME .
- docker push $IMAGE_NAME
only:
- main
run-tests:
stage: test
image: python:3.9-slim
script:
- echo "Running unit tests..."
- pip install -r requirements.txt
- pip install pytest
- pytest tests/unit/
only:
- main
deploy-staging:
stage: deploy
image: alpine:latest
script:
- echo "Deploying to staging..."
- apk add --no-cache curl
- |
curl --request POST \
--url "https://my-staging-deployment-api.example.com/deploy" \
--header "Content-Type: application/json" \
--data '{
"image_url": "'"$IMAGE_NAME"'"
}'
only:
- main
when: manual # Require manual trigger for staging deployment
This .gitlab-ci.yml defines three stages: build, test, and deploy. Each stage contains one or more jobs.
The docker-build job uses the official Docker image and a Docker-in-Docker (dind) service. It logs into the GitLab Container Registry using predefined CI variables (CI_REGISTRY_USER, CI_REGISTRY_PASSWORD, DOCKER_REGISTRY), builds the Docker image tagged with the commit SHA, and pushes it. This job runs only on pushes to the main branch.
The run-tests job uses a Python 3.9 image. It installs dependencies and runs pytest against the unit tests located in tests/unit/. This also runs only on pushes to main.
The deploy-staging job, again on main, is a placeholder. It simulates a deployment by using curl to POST a request to a hypothetical staging deployment API, passing the Docker image URL. The when: manual directive means this job won’t run automatically; it requires a user to click a "play" button in the GitLab UI.
The core idea is that each job is an isolated container. The image keyword specifies the Docker image to use for that job’s environment. services allows you to spin up additional containers that your job container can communicate with, like the docker:dind service for Docker commands.
The script section is where the magic happens – a series of shell commands executed sequentially. only and except keywords control which branches or tags trigger specific jobs. stages define the order of execution. Jobs within the same stage run in parallel if you have enough runners, while stages run sequentially.
To make this work, you need a GitLab Runner configured for your project or group. The runner is the agent that picks up jobs from GitLab CI and executes them. For Docker builds, you’ll typically want a runner with the Docker executor.
When a commit is pushed to main, GitLab CI triggers the pipeline. The docker-build job starts, followed by run-tests. If both succeed, the deploy-staging job becomes available for manual triggering. The IMAGE_NAME variable, which includes the short SHA of the commit, ensures that each deployment is tied to a specific code version.
The only: [main] clause is crucial for preventing accidental deployments from feature branches. You might have separate pipelines or jobs for feature branches that only perform builds and tests.
This setup ensures that your microservice is built into a reproducible Docker image, tested thoroughly, and then deployed in a controlled manner. The manual trigger for staging adds a layer of safety, preventing automated deployments to environments that might impact users.
The real power comes when you start adding more complex logic, like deploying to different environments (staging, production), running integration tests, performing security scans, or managing database migrations, all within this declarative YAML structure. Each job can have its own environment, dependencies, and execution logic, allowing for highly customized pipelines tailored to your microservice’s specific needs.
You’ll notice that the deploy-staging job uses a manual trigger. This is a common practice to add a human gate before deploying to environments that affect users.
The next logical step is to automate deployments to production, perhaps based on successful staging deployments or manual approval from a specific user or group.