GitLab’s Secret Detection is surprisingly good at finding credentials, API keys, and other sensitive information that has accidentally been committed to your repository, but it’s not magic; it works by matching patterns, and sophisticated attackers can bypass it if you don’t understand its limitations.
Let’s see it in action. Imagine you have a .env file in your repository that looks like this:
DATABASE_URL=postgres://user:password123@host.docker.internal:5432/mydb
STRIPE_SECRET_KEY=sk_test_abcdefghijklmnopqrstuvwxyz0123456789
AWS_ACCESS_KEY_ID=AKIAIOSF4EPYZXDJXQR4
AWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
When this file is committed and a CI/CD pipeline runs with Secret Detection enabled, GitLab will flag these lines. In the pipeline’s job output, you’ll see something like this:
SECRET DETECTED: DATABASE_URL found in .env
SECRET DETECTED: STRIPE_SECRET_KEY found in .env
SECRET DETECTED: AWS_ACCESS_KEY_ID found in .env
SECRET DETECTED: AWS_SECRET_ACCESS_KEY found in .env
This is GitLab’s default behavior, and it’s incredibly useful for catching common mistakes.
The Problem It Solves
The core problem Secret Detection addresses is the accidental exposure of credentials. Developers often store API keys, database passwords, and other secrets directly in their codebase for convenience during development. However, when this code is pushed to a version control system like GitLab, these secrets become part of the repository’s history, accessible to anyone with read access. This can lead to unauthorized access, data breaches, and significant financial or reputational damage.
How It Works Internally
GitLab Secret Detection, powered by the open-source project gitleaks, operates by scanning the content of your repository’s files and commit history. It uses a predefined set of regular expressions and entropy-based checks to identify potential secrets.
- Regular Expressions: These are patterns designed to match the typical format of known secrets, such as AWS access keys, private SSH keys, API tokens, and more. For example, a common pattern for AWS secret access keys looks for strings starting with
AKIAfollowed by a mix of uppercase letters, numbers, and then a signature-like string. - Entropy Analysis: For strings that don’t match a specific known pattern, Secret Detection can perform entropy analysis. This measures the randomness of a string. Highly random strings are more likely to be secrets (like randomly generated API keys) than predictable strings. If a string’s entropy exceeds a certain threshold, it’s flagged as a potential secret.
This scanning happens automatically as part of your CI/CD pipeline. You don’t need to explicitly run a command in your local environment; GitLab handles it for you.
Controlling the Scan
You can configure Secret Detection through your .gitlab-ci.yml file. Here’s a basic example of how to include it:
include:
- template: Security/Secret-Detection.gitlab-ci.yml
# You can also configure gitleaks directly if you need more advanced options
# secret_detection:
# image: registry.gitlab.com/security-products/secret-detection:latest
# script:
# - gitleaks detect --source . --report-path gl-secret-detection-report.json --verbose
# artifacts:
# reports:
# secret_detection: gl-secret-detection-report.json
The include statement is the simplest way to enable it. GitLab provides a template that sets up the job for you.
For more fine-grained control, you can define your own secret_detection job. This allows you to specify:
image: The Docker image to use, typicallyregistry.gitlab.com/security-products/secret-detection:latest.script: The command to run.gitleaks detectis the core command.--source .: Specifies the directory to scan (the current directory of the repository).--report-path gl-secret-detection-report.json: Defines where to save the scan results. This is crucial for GitLab to process the report.--verbose: Provides more detailed output during the scan.
artifacts:reports:secret_detection: This tells GitLab to treat the specified JSON file as a secret detection report, enabling it to display findings in the UI.
You can also create a .gitleaks.toml file in your repository’s root to customize gitleaks behavior further. This file allows you to:
- Add custom rules: Define new regex patterns for secrets specific to your organization.
- Exclude files/directories: Prevent scanning of certain paths where secrets are legitimately expected or known to exist (e.g., a file used for encrypted secrets management).
- Modify detection thresholds: Adjust the entropy level required to flag a string as a secret.
Here’s an example .gitleaks.toml:
[allowlist]
# Ignore secrets in test data files
[allowlist.paths]
regex = '''(?i)(test/fixtures/.*\.json)'''
# You can also use glob patterns
# glob = "test/fixtures/*.json"
[[rules]]
id = "custom-api-key"
description = "Custom API Key for internal service"
regex = '''(?i)X-INTERNAL-API-KEY:\s*[a-zA-Z0-9+/=]{20,}'''
entropy = 3.0
tags = ["api_key", "internal"]
This configuration tells gitleaks to ignore files in test/fixtures/ and to look for a specific internal API key format.
The Nuance of Entropy
What many people don’t realize is that the entropy check is a powerful, albeit sometimes noisy, component. It’s designed to catch randomly generated strings that look like secrets, even if they don’t match a known pattern. This is why you might occasionally see legitimate-looking random strings flagged. The default entropy threshold is tuned to balance finding real secrets with minimizing false positives, but it’s not perfect. Understanding this helps you calibrate your expectations and potentially adjust the threshold in your .gitleaks.toml if you’re seeing too many or too few alerts.
When Secret Detection flags a finding, it shows up in the Merge Request widget and the Security tab of your project. You can then review these findings, verify if they are actual secrets, and either remove them from history or mark them as false positives.
The next step after fixing committed secrets is often to implement preventative measures like pre-commit hooks or more robust secret management solutions.