GCP Service Account keys are a common vector for compromise because they grant broad access to your cloud resources, and a compromised key often means a compromised account.

Let’s break down how to manage these keys securely, not just avoiding the obvious pitfalls, but digging into the more subtle but critical aspects.

The Core Problem: Long-Lived, Powerful Credentials

Service account keys are essentially long-lived passwords for your applications. When an attacker gets their hands on one, they can impersonate your service account and access anything it has permission for. The default behavior of creating and managing these keys can be risky.

1. Minimize Key Usage: The Golden Rule

  • Diagnosis: Audit your service accounts for keys that are actively being used and are not essential. Look for service accounts with keys that haven’t been rotated in a long time or are associated with applications that can be refactored to use Workload Identity.
  • Check:
    gcloud iam service-accounts keys list --iam-account SERVICE_ACCOUNT_EMAIL
    
    This command lists all keys for a given service account. Pay attention to the validAfterTime and validBeforeTime to understand their lifecycle.
  • Fix: Delete unnecessary keys.
    gcloud iam service-accounts keys delete KEY_ID --iam-account SERVICE_ACCOUNT_EMAIL
    
    Why it works: Reducing the number of active keys directly reduces the attack surface. Fewer keys mean fewer potential entry points for attackers.

2. Rotate Keys Regularly: Treat Them Like Passwords

  • Diagnosis: Identify service account keys that have been active for longer than your organization’s policy (e.g., 90 days). This is often a manual process or requires custom scripting if not using automated rotation.
  • Check: Use the output from the gcloud iam service-accounts keys list command to identify keys approaching or exceeding their rotation period.
  • Fix: Create a new key and then delete the old one.
    # Create a new key
    gcloud iam service-accounts keys create /tmp/new_key.json --iam-account SERVICE_ACCOUNT_EMAIL
    
    # Update your application to use the new key
    # Once confirmed, delete the old key
    gcloud iam service-accounts keys delete OLD_KEY_ID --iam-account SERVICE_ACCOUNT_EMAIL
    
    Why it works: Regular rotation ensures that even if a key is compromised, its lifespan in an attacker’s hands is limited, minimizing the window of opportunity for abuse.

3. Use Workload Identity Federation (for GKE): The Modern Approach

  • Diagnosis: You’re manually managing service account keys for applications running on Google Kubernetes Engine (GKE). This is inherently insecure as keys need to be distributed to pods.
  • Check: Review your GKE deployments. If you’re mounting service account key files into pods, you’re doing it wrong.
  • Fix: Configure Workload Identity. This allows GKE workloads to impersonate Google Cloud service accounts without needing any long-lived credentials.
    1. Enable Workload Identity on your GKE cluster:
      gcloud container clusters update CLUSTER_NAME \
          --zone ZONE \
          --workload-pool PROJECT_ID.svc.id.goog
      
    2. Annotate your Kubernetes service account:
      kubectl annotate serviceaccount KSA_NAME \
          iam.gke.io/gcp-service-account=GCP_SERVICE_ACCOUNT_EMAIL
      
    3. Grant the GKE workload’s identity permissions to the GCP service account. Why it works: Workload Identity uses short-lived, rotating tokens issued by the Kubernetes cluster’s identity provider, which are then exchanged for GCP credentials. No long-lived keys are ever exposed.

4. Limit Key Scope: Principle of Least Privilege

  • Diagnosis: Service accounts have overly broad IAM roles assigned to them, granting them access to more resources than they strictly need. This is often due to convenience during initial setup.
  • Check: Review the IAM policies attached to your service accounts.
    gcloud projects get-iam-policy PROJECT_ID --flatten="bindings[].members" --filter="bindings.members:serviceAccount:SERVICE_ACCOUNT_EMAIL"
    
  • Fix: Create custom IAM roles with only the necessary permissions.
    1. Define your custom role in a YAML file (e.g., custom-role.yaml):
      title: "My Custom Role"
      description: "Grants specific permissions for my application."
      stage: "GA"
      includedPermissions:
      - storage.objects.get
      - storage.objects.list
      
    2. Create the custom role:
      gcloud iam roles create myCustomRole --project=PROJECT_ID --file=custom-role.yaml
      
    3. Grant this custom role to the service account:
      gcloud projects add-iam-policy-binding PROJECT_ID \
          --member="serviceAccount:SERVICE_ACCOUNT_EMAIL" \
          --role="projects/PROJECT_ID/roles/myCustomRole"
      
    Why it works: By adhering to the principle of least privilege, you significantly reduce the potential damage an attacker can inflict if they manage to compromise a service account key or the service account itself.

5. Distribute Keys Securely: Avoid Hardcoding

  • Diagnosis: Service account keys are embedded directly into application code, configuration files, or source control repositories.
  • Check: Static analysis of your codebase and configuration management tools. Look for .json files containing key credentials.
  • Fix: Use a secrets management solution like Google Secret Manager or HashiCorp Vault.
    1. Store the key in Secret Manager:
      echo "YOUR_KEY_JSON_CONTENT" | gcloud secrets create my-service-account-key --data-file=-
      
    2. Grant the service account (or the application running as it) permission to access the secret.
    3. Have your application retrieve the secret at runtime. Why it works: Secrets management tools are designed for secure storage and retrieval of sensitive information, preventing accidental exposure through code commits or insecure file permissions.

6. Monitor Key Usage: Detect Anomalies

  • Diagnosis: You lack visibility into when and where your service account keys are being used, making it hard to spot suspicious activity.
  • Check: Enable and review Cloud Audit Logs for "Admin Activity" and "Data Access" logs, filtering for service account key usage.
  • Fix: Configure Cloud Logging to export relevant logs to BigQuery for analysis and set up alerts.
    1. Create a log sink:
      gcloud logging sinks create service-account-key-usage-sink \
          logging.googleapis.com/projects/PROJECT_ID/sinks/service-account-key-usage-sink \
          --log-filter='protoPayload.methodName="google.iam.admin.v1.IAMCredentials.GenerateAccessToken" OR protoPayload.methodName="google.iam.admin.v1.IAMCredentials.GenerateIdToken"' \
          --destination=bigquery.googleapis.com/projects/PROJECT_ID/datasets/audit_logs_dataset
      
    2. Create alerts based on unusual patterns in BigQuery. Why it works: Continuous monitoring allows for the early detection of anomalous key usage patterns, which could indicate a compromise, enabling a faster response.

The next hurdle you’ll face is managing the lifecycle of these keys across a large, dynamic environment, which often leads to exploring automated rotation and policy enforcement tools.

Want structured learning?

Take the full Gcp course →