Linkerd Prometheus: Scrape and Alert on Mesh Metrics (2026)

Linkerd’s Prometheus integration is surprisingly opaque, but it’s not about Prometheus pulling data; it’s about Linkerd exposing it and Prometheus scraping it, which sounds similar but is fundamentally different for how you’d debug.

Let’s see Linkerd’s metrics in action.

# First, ensure you have Linkerd installed and running
linkerd install | kubectl apply -f -

# Wait for the control plane to be ready
kubectl get pods -n linkerd -w

# Now, let's simulate some traffic to a sample application
# Deploy the emojivoto application
kubectl apply -f https://raw.githubusercontent.com/linkerd/linkerd2/main/examples/emojivoto/emojivoto.yml

# Inject the Linkerd proxy into the emojivoto pods
linkerd inject --manual emojivoto/voting | kubectl apply -f -

# Generate some traffic
for i in {1..100}; do \
  curl -sS "http://voting.emojivoto.svc.cluster.local/vote/red" > /dev/null; \
  curl -sS "http://voting.emojivoto.svc.cluster.local/vote/blue" > /dev/null; \
  curl -sS "http://voting.emojivoto.svc.cluster.local/vote/green" > /dev/null; \
  curl -sS "http://voting.emojivoto.svc.cluster.local/vote/yellow" > /dev/null; \
  sleep 0.1; \
done

# Now, let's find the Prometheus endpoint for Linkerd
# Linkerd's Prometheus metrics are exposed by the 'linkerd-proxy' container
# within each application pod. The default scrape port is 4142.

# We can use kubectl port-forward to access one of these proxies directly
# Let's pick a voting pod as an example
kubectl -n emojivoto port-forward deploy/voting 4142:4142

# In another terminal, use curl to scrape the metrics
curl http://localhost:4142/metrics | grep "linkerd_"

# You'll see output like this (truncated):
# HELP linkerd_proxy_success_total Total number of successful requests.
# TYPE linkerd_proxy_success_total counter
# HELP linkerd_proxy_request_duration_seconds Distribution of request durations.
# TYPE linkerd_proxy_request_duration_seconds histogram
# HELP linkerd_proxy_tcp_open_total Total number of TCP connections opened.
# TYPE linkerd_proxy_tcp_open_total counter
# ... and many more metrics related to requests, latency, TCP connections, etc.

The core problem Linkerd’s metrics solve is providing a standardized, application-agnostic way to observe the behavior of your microservices at the network layer. It doesn’t care if your service is written in Go, Python, or Java; the Linkerd proxy intercepts all HTTP and TCP traffic and reports on it. This means you can get insights into request success rates, latency distributions, and connection counts without modifying your application code.

Internally, each Linkerd proxy runs an embedded Prometheus exposition endpoint. When you configure Prometheus to scrape this endpoint (typically on port 4142 within the application pod), Prometheus collects these metrics. Linkerd’s control plane components (like linkerd-controller and linkerd-destination) also expose their own Prometheus metrics, usually on port 9990.

The key levers you control are:

1. Scraping Configuration: How you tell Prometheus to find and collect metrics from the Linkerd proxies. This involves ServiceMonitor or PodMonitor resources in Prometheus Operator environments, or static configuration in standalone Prometheus.
2. Metric Granularity: Linkerd provides a rich set of metrics. You can choose which ones are most important for your alerting and dashboarding needs.
3. Alerting Rules: Defining PrometheusRule resources to trigger alerts based on specific metric thresholds (e.g., high error rates, increased latency).
4. Proxy Configuration: While less common for basic metrics, you can fine-tune proxy behavior (e.g., tracing, TLS) which might indirectly affect metrics.

The most surprising thing about Linkerd’s Prometheus metrics is their sheer volume and the fact that they are generated by the proxy, not the application. This means you get consistent, detailed metrics across all your services without any application code changes. For example, linkerd_proxy_request_duration_seconds is a histogram that provides percentiles (like p50, p95, p99) of request latency, offering a much richer view than a simple average.

When you configure Prometheus to scrape the linkerd-proxy sidecars, it’s not just fetching simple counters. Prometheus interprets the histogram and summary metric types to calculate quantiles, which are crucial for understanding tail latencies. This means your Prometheus setup is already capable of calculating p99 latency from the data Linkerd provides, without you needing to explicitly sum or average anything.