The most surprising thing about routing GCP logs is that you’re not just copying logs; you’re essentially creating a real-time data pipeline from your entire GCP environment.
Let’s see this in action. Imagine you’re running a web application on GKE, and you want to capture all incoming requests to analyze them later, store them for compliance, and alert on suspicious activity.
First, we need to create a sink. This is the core component that defines what logs to capture and where to send them.
gcloud logging sinks create my-log-sink \
logging.googleapis.com/projects/my-gcp-project \
--destination="bigquery.googleapis.com/projects/my-gcp-project/datasets/my_log_dataset" \
--log-filter='resource.type="k8s_container" AND resource.labels.cluster_name="my-gke-cluster"' \
--use-partitioned-tables
This command creates a sink named my-log-sink. It targets logs within my-gcp-project. The --destination points to a BigQuery dataset named my_log_dataset. The --log-filter is crucial: it specifies that we only want logs from Kubernetes containers within a cluster named my-gke-cluster. Finally, --use-partitioned-tables tells BigQuery to create daily partitions for better query performance and cost management, which is a must-have for large log volumes.
Once this sink is created, GCP automatically provisions a service account for it. This service account needs permissions to write to the destination. For BigQuery, it needs roles/bigquery.dataEditor. For Cloud Storage, roles/storage.objectCreator. For Pub/Sub, roles/pubsub.publisher.
Here’s how you’d grant that permission for our BigQuery example:
# Get the service account email for the sink
SINK_SA=$(gcloud logging sinks describe my-log-sink --format='value(writerIdentity)')
# Grant BigQuery data editor role to the sink's service account
gcloud projects add-iam-policy-binding my-gcp-project \
--member="${SINK_SA}" \
--role="roles/bigquery.dataEditor"
Now, logs matching our filter will start flowing into my_log_dataset in BigQuery. You can query them directly:
SELECT
timestamp,
jsonPayload.message,
resource.labels.pod_name
FROM
`my-gcp-project.my_log_dataset.k8s_container_my-gke-cluster_*` -- The wildcard matches daily partitions
WHERE
resource.type = 'k8s_container'
ORDER BY
timestamp DESC
LIMIT 1000
But what if you also want to archive these logs to Cloud Storage for long-term, cost-effective storage, and stream them to Pub/Sub for immediate alerting? You create additional sinks.
# Sink to Cloud Storage
gcloud logging sinks create my-log-archive-sink \
logging.googleapis.com/projects/my-gcp-project \
--destination="storage.googleapis.com/my-log-bucket/gcp-logs/" \
--log-filter='resource.type="k8s_container" AND resource.labels.cluster_name="my-gke-cluster"'
# Grant Cloud Storage permission
GCS_SINK_SA=$(gcloud logging sinks describe my-log-archive-sink --format='value(writerIdentity)')
gsutil iam ch serviceAccount:${GCS_SINK_SA}:objectCreator gs://my-log-bucket
# Sink to Pub/Sub
gcloud logging sinks create my-log-alert-sink \
logging.googleapis.com/projects/my-gcp-project \
--destination="pubsub.googleapis.com/projects/my-gcp-project/topics/my-log-alerts" \
--log-filter='resource.type="k8s_container" AND resource.labels.cluster_name="my-gke-cluster" AND httpRequest.status >= 500'
# Grant Pub/Sub permission
PUBSUB_SINK_SA=$(gcloud logging sinks describe my-log-alert-sink --format='value(writerIdentity)')
gcloud projects add-iam-policy-binding my-gcp-project \
--member="${PUBSUB_SINK_SA}" \
--role="roles/pubsub.publisher"
Notice the filter for the Pub/Sub sink: httpRequest.status >= 500. This demonstrates how you can create highly specific filters for different destinations. You’re not limited to sending all logs everywhere.
The mental model is this: Logs are generated by GCP resources. The Logging agent (or service) collects them. Log sinks are configured to subscribe to a filtered stream of these logs and then publish them to a destination. The writerIdentity is the key: it’s the service account that acts on behalf of the sink, carrying the necessary permissions to write to your chosen endpoints.
The most powerful aspect, and often overlooked, is that log sinks don’t just capture logs from the project they are created in. You can create a sink in a dedicated logging project and use the logging.googleapis.com/projects/[PROJECT_ID] format in the --destination to route logs from other projects into a centralized logging bucket or BigQuery dataset. This is how you achieve a truly unified view of your GCP logging across multiple projects without needing to manage sinks in each individual project.
The next step is often setting up log-based metrics from your BigQuery data to trigger those Pub/Sub alerts or building complex dashboards in Looker Studio from your BigQuery logs.