Collect Istio Metrics with Prometheus (2026)

Prometheus doesn’t actually "collect" metrics; it scrapes them from endpoints that expose them in a specific format.

Let’s see Istio’s metrics in action.

First, ensure you have Istio installed and a sample application running within the mesh. A common test is bookinfo. If you don’t have it, install it:

kubectl apply -f samples/bookinfo/platform/kube/bookinfo.yaml
kubectl apply -f samples/bookinfo/networking/bookinfo-gateway.yaml

Now, let’s poke around. You’ll need to expose Prometheus, which is usually deployed as part of Istio. The default service name is prometheus-istio.

kubectl port-forward -n istio-system service/prometheus-istio 9090:9090

Open your browser to http://localhost:9090/graph. In the expression bar, type istio_requests_total. You should see a stream of metrics, like this:

{
  "status": "success",
  "data": {
    "resultType": "vector",
    "result": [
      {
        "metric": {
          "__name__": "istio_requests_total",
          "destination_canonical_service": "details",
          "destination_canonical_service_namespace": "default",
          "destination_principal": "cluster.local/sa/default",
          "destination_service": "details.default.svc.cluster.local",
          "destination_service_name": "details",
          "destination_workload": "details-v1",
          "destination_workload_namespace": "default",
          "instance": "istio-ingressgateway-5d57d7b8b8-abcd",
          "job": "istio-ingressgateway",
          "reporter": "destination",
          "request_protocol": "http",
          "response_code": "200",
          "response_flags": "-",
          "source_canonical_service": "productpage",
          "source_canonical_service_namespace": "default",
          "source_principal": "cluster.local/sa/default",
          "source_workload": "productpage-v1",
          "source_workload_namespace": "default"
        },
        "value": [
          1678886400,
          "12345"
        ]
      },
      // ... more metrics
    ]
  }
}

This istio_requests_total metric is a counter that increments with every request processed by the Istio sidecar proxies (Envoy). The labels provide rich context: source_workload, destination_workload, response_code, request_protocol, and so on.

The Problem Istio Metrics Solve:

Istio’s primary goal is to provide observability, traffic management, and security for microservices without requiring application code changes. Before Istio, getting detailed, consistent metrics across all your services was a nightmare. Each service might have its own metrics library, format, and level of detail. Istio’s sidecar proxies intercept all ingress and egress traffic, allowing them to generate standardized metrics for every request/response.

How it Works Internally:

1. Envoy Proxies: Istio injects Envoy sidecar proxies alongside your application pods.
2. Traffic Interception: Envoy intercepts all inbound and outbound network traffic for the pod.
3. Metric Generation: As traffic flows through Envoy, it generates a predefined set of metrics (e.g., istio_requests_total, istio_request_duration_milliseconds). These metrics are exposed over an HTTP endpoint, typically /stats/prometheus, on the Envoy proxy itself (usually on port 15020).
4. Prometheus Scrape Config: Prometheus is configured to scrape these /stats/prometheus endpoints from all Envoy proxies within the cluster. Istio’s Prometheus installation already has this configuration set up, targeting the Envoy sidecars.
5. Prometheus Storage & Querying: Prometheus stores these time-series metrics and provides a powerful query language (PromQL) to analyze them.

Key Levers You Control:

• Scrape Interval: The frequency at which Prometheus fetches metrics from the Envoy endpoints. This is configured in Prometheus’s scrape_configs. A shorter interval gives more granular data but increases load on Prometheus and the targets.
• Metric Filtering/Relabeling: Prometheus can filter or modify labels of metrics before they are stored. This is done via relabel_configs and metric_relabel_configs. For instance, you might want to drop metrics from specific namespaces or add custom labels.
• Retention Period: How long Prometheus keeps the collected data. Configured via --storage.tsdb.retention.time in Prometheus.
• Istio Proxy Configuration: While not directly Prometheus config, you can influence what metrics Envoy generates. For example, you can disable certain metrics or configure the stats port if needed, though this is rarely necessary.

What people often miss is that the Istio sidecar proxy doesn’t just send metrics to Prometheus; it exposes them on a local HTTP endpoint. Prometheus then pulls these metrics from that endpoint. This pull model is fundamental to Prometheus and allows for better control over scrape frequency and error handling compared to a push model where targets send data. The job label in the Prometheus UI often reflects the name of the scrape configuration that matched the target, which in Istio’s case typically points to the Envoy sidecar.

The next challenge is usually correlating these raw metrics with specific application behavior or setting up alerting based on them.