Kafka brokers are rejecting client connections due to failed SASL authentication, preventing any client from producing or consuming messages.

Common Causes and Fixes for Kafka SASL Authentication Failures

1. Incorrect sasl.enabled Configuration

  • Diagnosis: Check server.properties on your Kafka brokers. Ensure sasl.enabled is set to true. If it’s false, SASL is disabled, and clients configured for SASL will fail.
  • Fix: Set sasl.enabled=true in server.properties on all brokers.
  • Why it works: This explicitly tells the Kafka broker to expect and process SASL authentication mechanisms for incoming connections.

2. Mismatched sasl.mechanism

  • Diagnosis: Verify the sasl.mechanism setting in both your client configuration (e.g., producer.properties, consumer.properties) and your broker configuration (server.properties). Common mechanisms are PLAIN, GSSAPI (Kerberos), and SCRAM-SHA-256/SCRAM-SHA-512.
  • Fix: Ensure the sasl.mechanism value is identical on both the client and broker. For example, if your client has sasl.mechanism=PLAIN, your broker must also have sasl.mechanism=PLAIN (or be configured to support it via sasl.mechanism.list).
  • Why it works: The client and broker must agree on the authentication protocol to use. A mismatch means they can’t understand each other’s authentication handshake.

3. Incorrect sasl.jaas.config on Brokers

  • Diagnosis: Examine the sasl.jaas.config property in server.properties on your Kafka brokers. This file specifies the JAAS (Java Authentication and Authorization Service) configuration for Kafka. Common issues include typos, incorrect file paths for keytabs/principals, or wrong realm names.
  • Fix: For Kerberos (GSSAPI), ensure the sasl.jaas.config points to the correct keytab file and principal. For example:
    sasl.jaas.config=com.sun.security.auth.module.Krb5LoginModule required \
    useKeyTab=true \
    storeKey=true \
    keyTab="/etc/kafka/kafka.service.keytab" \
    principal="kafka/kafka-broker.example.com@EXAMPLE.COM";
    
    For PLAIN or SCRAM, ensure the login.controller.factory and relevant users configurations are correct if you’re using Kafka’s internal user store.
  • Why it works: JAAS is the underlying framework Kafka uses for authentication. Correctly configured JAAS entries tell Kafka how to find and validate credentials (like Kerberos tickets or username/passwords).

4. Incorrect sasl.jaas.config on Clients

  • Diagnosis: Inspect the sasl.jaas.config in your client properties files. This is particularly relevant for Kerberos. Ensure the client’s JAAS configuration correctly identifies its principal and keytab, and that the Kerberos realm (.realm) and KDC (.kdc) are correctly specified if not using a default krb5.conf.
  • Fix: For Kerberos clients, a typical sasl.jaas.config looks like this:
    sasl.jaas.config=com.sun.security.auth.module.Krb5LoginModule required \
    useKeyTab=true \
    storeKey=true \
    keyTab="/path/to/client.keytab" \
    principal="client-principal@EXAMPLE.COM";
    
    If using PLAIN or SCRAM, this might contain username/password information, but it’s generally better to use environment variables or external credential providers for security.
  • Why it works: Clients need their own valid credentials and configuration to authenticate to the brokers. The client’s JAAS config dictates how it obtains and presents its identity.

5. Time Skew (Kerberos Specific)

  • Diagnosis: If using Kerberos (GSSAPI), significant time differences between the Kafka brokers, the client machines, and the Kerberos Key Distribution Center (KDC) can cause authentication failures. Check the system time on all relevant machines.
  • Fix: Ensure all servers (brokers, clients, KDC) are synchronized using Network Time Protocol (NTP). The maximum allowable skew is typically 5 minutes.
  • Why it works: Kerberos tickets have a limited lifespan and are time-sensitive. If the clocks are too far apart, the ticket presented by the client will appear expired or invalid to the KDC or broker.

6. Missing or Incorrect Principal Mapping

  • Diagnosis: For mechanisms like GSSAPI (Kerberos) or SCRAM, the broker needs to know how to map the authenticated principal (e.g., kafka/broker.host@REALM) to a Kafka user. This is often configured via principal.mapper.rules in server.properties. Check if the rule correctly extracts the username from the authenticated principal.
  • Fix: Update principal.mapper.rules to correctly map the incoming principal to a Kafka user. For example, to map a Kerberos principal user/host@REALM to a Kafka user user:
    principal.mapper.rules=RULE:^User:([a-zA-Z0-9._/-]+)@.*$/$1/
    
    Ensure the Kafka user ($1 in this example) exists in your authorization configuration if authorization is also enabled.
  • Why it works: Authentication confirms who the client is, but mapping translates that identity into a Kafka-internal user that Kafka’s authorization layer can understand and use.

7. Incorrect File Permissions for Keytabs

  • Diagnosis: For Kerberos, if the keyTab file specified in sasl.jaas.config is not readable by the Kafka process user (e.g., kafka), authentication will fail.
  • Fix: Ensure the Kafka user has read permissions on the keytab file. For example, if your keytab is at /etc/kafka/kafka.service.keytab and Kafka runs as user kafka and group kafka:
    sudo chown kafka:kafka /etc/kafka/kafka.service.keytab
    sudo chmod 600 /etc/kafka/kafka.service.keytab
    
  • Why it works: The Kafka process needs to access the keytab file to load its Kerberos credentials and authenticate itself as a service principal to the KDC.

The next error you’ll likely encounter, after fixing authentication, is an authorization error if your ACLs are not correctly configured.

Want structured learning?

Take the full Kafka course →