Kafka Prometheus Monitoring: Export JMX Metrics (2026)

Kafka’s Prometheus monitoring, specifically exporting JMX metrics, is more about translating Kafka’s internal state into Prometheus’s pull-based model than about Prometheus itself.

Let’s watch it run. Imagine you’ve got a Kafka broker humming along, spewing metrics from its Java Management Extensions (JMX) interface. Prometheus, on the other hand, likes to scrape HTTP endpoints. The bridge between these two worlds is the jmx_exporter.

Here’s a simplified kafka-jmx-exporter.yml configuration:

---
lowercaseOutputName: true
lowercaseOutputLabelNames: true
whitelistObjectNames:
  - "kafka.server:type=BrokerTopicMetrics,name=*"
  - "kafka.controller:type=KafkaController,name=*"
rules:
  - pattern: kafka.server<type=BrokerTopicMetrics, name=(.+)><>Value
    name: kafka_server_brokertopicmetrics_$1_value
  - pattern: kafka.controller<type=KafkaController, name=(.+)><>Value
    name: kafka_controller_kafkaproxy_$1_value

This configuration tells the jmx_exporter to:

• Convert metric names and labels to lowercase.
• Only collect metrics related to BrokerTopicMetrics and KafkaController.
• Transform specific JMX patterns into Prometheus-friendly metric names. For example, kafka.server<type=BrokerTopicMetrics, name=BytesInPerSec><>Value becomes kafka_server_brokertopicmetrics_bytesinpersec_value.

You’d run this exporter as a Java agent attached to your Kafka broker JVM. The command line for your Kafka broker might look something like this:

/usr/bin/java \
  -Xmx1G \
  -Xms1G \
  -javaagent:/path/to/jmx_exporter.jar=9404:/path/to/kafka-jmx-exporter.yml \
  -cp ... \
  kafka.Kafka \
  /path/to/server.properties

Here, 9404 is the port the jmx_exporter will listen on for Prometheus scrapes. The kafka-jmx-exporter.yml is the rules file we saw earlier.

Once running, Prometheus can be configured to scrape http://<kafka-broker-ip>:9404/metrics. This allows Prometheus to pull all the metrics you’ve whitelisted and transformed from the Kafka broker’s JMX interface.

The core problem this solves is bridging Kafka’s internal, Java-centric monitoring (JMX) with Prometheus’s external, HTTP-based scraping model. Without the jmx_exporter, Prometheus wouldn’t know how to ask Kafka for its metrics. The exporter acts as a translator, exposing JMX MBeans over HTTP in a format Prometheus understands.

Internally, the jmx_exporter uses the Java Management Extensions API to query MBeans exposed by Kafka. It then applies your defined rules to transform the JMX object names and attribute values into Prometheus metric names and labels. These transformed metrics are then served over an HTTP endpoint.

The exact levers you control are primarily in the jmx_exporter configuration file. You define which JMX object names to collect (whitelistObjectNames), and how to translate their attributes into Prometheus metrics (rules). This includes mapping JMX property names to Prometheus label names and JMX attribute values to Prometheus metric values. You also control the port the exporter listens on and which JVMs it’s attached to.

A common misconception is that you need to configure Kafka itself to expose Prometheus metrics. Kafka doesn’t natively do this; it relies on external tools like the jmx_exporter to make its JMX data accessible to Prometheus.

The next concept you’ll likely run into is setting up alerting rules in Prometheus based on these scraped Kafka metrics.