Fix Kafka ClusterAuthorizationFailed Errors (2026)

The Kafka authorization system is failing because the client’s identity isn’t being recognized or is being rejected by the broker. This happens when a client tries to perform an operation (like producing or consuming) and the broker’s authorization layer checks its credentials against the configured access control lists (ACLs) and finds no match or an explicit deny.

Common Causes and Fixes for AuthorizationFailed

1. Incorrect Principal in ACLs: The most frequent culprit is a mismatch between the principal (user/identity) specified in the ACL and the principal the client is actually presenting. Kafka ACLs are very specific about who can do what.

   • Diagnosis: Check the ACLs for the resource the client is trying to access.

     kafka-acls.sh --bootstrap-server localhost:9092 --list --topic my-topic
     

     Compare the output principal (e.g., User:my-app-user) with the principal your client is configured to use. If your client is configured to use Kerberos, the principal will look like User:myapp/hostname@REALM. If it’s SSL, it will be the distinguished name from the certificate (e.g., User:CN=my-app-client,OU=IT,O=MyOrg,L=City,ST=State,C=US).
   • Fix: Add or modify the ACL to include the correct principal. For example, to grant my-app-user read and write access to my-topic:

     kafka-acls.sh --bootstrap-server localhost:9092 --add --topic my-topic --producer --consumer --group consumer-group-id --acl-principal-type User --acl-principal-name my-app-user
     

     If using Kerberos, ensure the principal is correctly formatted and exists in your KDC. If using SSL, ensure the client certificate’s Subject DN matches what’s in the ACL.
   • Why it works: The broker looks for an exact match of the client’s authenticated identity against the principals listed in the ACL rules. If the principal name or type (User, Group, DN, etc.) doesn’t match, authorization fails.

2. Client Not Authenticating Correctly: The client might be failing to authenticate with the broker in the first place, meaning it doesn’t have a valid identity for the broker to check against ACLs. This is common with SSL/TLS or Kerberos.

   • Diagnosis (SSL): Check Kafka broker logs (server.log) for SSL handshake errors, certificate validation failures, or messages indicating an unknown client certificate. On the client side, check client logs for SSLHandshakeException or similar.
   • Fix (SSL): Ensure the client has the correct keystore and truststore configured, and that the client certificate is trusted by the Kafka brokers (i.e., its issuer is in the broker’s truststore). Verify client properties like ssl.truststore.location, ssl.truststore.password, ssl.keystore.location, ssl.keystore.password, and ssl.key.password.
     • Example client properties:

       ssl.truststore.location=/path/to/client.truststore.jks
       ssl.truststore.password=truststore_password
       ssl.keystore.location=/path/to/client.keystore.jks
       ssl.keystore.password=keystore_password
       ssl.key.password=key_password
       

   • Diagnosis (Kerberos): Check client logs for Kerberos-related errors (e.g., GSSException, JAAS configuration error). Ensure the client has a valid Kerberos ticket (klist).
   • Fix (Kerberos): Verify the kafka.security.auth.login.callback.handler.class and java.security.auth.login.config client properties are correctly set to point to your JAAS configuration file. Ensure the JAAS file correctly specifies the Kerberos realm and principal for the client.
     • Example JAAS config (client.jaas):

       KafkaClient {
           com.sun.security.auth.module.Krb5LoginModule required
           useKeyTab=false
           storeKey=false
           isInitiator=true
           principal="my-app-user@MY.REALM";
       };
       

     And the client property: properties java.security.auth.login.config=/path/to/client.jaas
   • Why it works: Authentication is the prerequisite for authorization. If the client can’t establish a secure identity with the broker, the broker has no "user" to check against ACLs, leading to a denial.

3. Incorrect authorizer.class.name in Broker Config: The broker might not be configured to use an authorizer at all, or it’s configured with an incorrect or incompatible authorizer class.

   • Diagnosis: Check the server.properties file on your Kafka brokers. Look for the authorizer.class.name property.
   • Fix: Ensure it’s set to a valid authorizer. The most common is kafka.security.authorizer.AclAuthorizer. If you’re using a custom authorizer, ensure the class is available on the broker’s classpath and correctly configured.

     # In server.properties
     authorizer.class.name=kafka.security.authorizer.AclAuthorizer
     

   • Why it works: Without a configured authorizer, the broker defaults to allowing all operations, or it might reject the request if the authorizer class is missing or invalid, preventing any authorization checks from happening.

4. ZooKeeper ACLs Blocking Kafka ACLs: If you are using ZooKeeper for Kafka ACL management (authorizer.class.name=kafka.security.authorizer.AclAuthorizer and zookeeper.connect=...), ZooKeeper itself can have ACLs that prevent the Kafka brokers from reading or writing ACLs.

   • Diagnosis: Check ZooKeeper’s ACLs for the Kafka ACL node (typically /kafka/acl or /acl).

     # Connect to ZK
     zkCli.sh -server <zookeeper_host>:2181
     
     # List ACLs for the ACL node
     getAcls /kafka/acl
     

     Look for permissions granted to the Kafka broker’s ZK principal.
   • Fix: Grant the necessary permissions (usually crdwa - create, read, delete, write, all) to the Kafka broker’s principal in ZooKeeper.

     # Example: Granting full permissions to "kafka_broker_user" on /kafka/acl
     setAcl /kafka/acl kafka_broker_user:rwcda,-1
     

     You’ll need to know the principal Kafka uses to connect to ZooKeeper. This is often configured via JAAS for the broker if Kerberos is used, or it might be a simple username if SASL/PLAIN or other simpler auth is used for ZK connections.
   • Why it works: The Kafka AclAuthorizer relies on ZooKeeper to store and retrieve ACLs. If the broker process cannot read from or write to the ACL node in ZooKeeper due to ZooKeeper’s own ACL restrictions, it cannot manage Kafka ACLs, leading to authorization failures.

5. Incorrect Principal for super.users: The super.users configuration in server.properties is a way to bypass ACL checks for specific principals. If the principal your client is using isn’t listed here and should be, it will be subject to normal ACL checks.

   • Diagnosis: Check server.properties for super.users.
   • Fix: Add the client’s principal to the super.users list. Note that this bypasses all ACL checks, so use it judiciously.

     # In server.properties
     super.users=User:kafka-admin-user,User:my-special-client
     

     Ensure the principal format matches exactly (e.g., User:my-app-user for SASL/PLAIN or User:my-app-user@MY.REALM for Kerberos).
   • Why it works: Super users are explicitly exempted from all ACL checks. If a user needs to perform operations that are otherwise denied by ACLs, but should be allowed without checking ACLs, adding them to super.users grants that privilege.

6. Topic/Group/Cluster-Level ACLs Missing or Incorrect: ACLs can be set at different levels: cluster-wide, topic, consumer group, and transaction. A client might be authorized for a topic but not for the consumer group it’s trying to join, or vice-versa.

   • Diagnosis: Use kafka-acls.sh --list for the specific resource. Pay attention to the resource type (TOPIC, GROUP, CLUSTER, TRANSACTION) and the operations granted (READ, WRITE, CREATE, DELETE, ALTER, DESCRIBE, CLUSTER_ACTION, AUTHORIZE_TRANSACTIONS).

     # Check topic ACLs
     kafka-acls.sh --bootstrap-server localhost:9092 --list --topic my-topic
     
     # Check consumer group ACLs
     kafka-acls.sh --bootstrap-server localhost:9092 --list --group my-consumer-group
     
     # Check cluster-wide ACLs (e.g., for DESCRIBE CLUSTER)
     kafka-acls.sh --bootstrap-server localhost:9092 --list --cluster
     

   • Fix: Add or modify ACLs to cover all necessary operations and resource types. For a consumer to read from my-topic using group my-consumer-group, it needs READ on my-topic and READ, DESCRIBE, OFFSET_RESET on my-consumer-group.

     # Grant read on topic
     kafka-acls.sh --bootstrap-server localhost:9092 --add --topic my-topic --consumer --acl-principal-type User --acl-principal-name my-app-user
     
     # Grant read/describe on group
     kafka-acls.sh --bootstrap-server localhost:9092 --add --group my-consumer-group --consumer --acl-principal-type User --acl-principal-name my-app-user
     

   • Why it works: Kafka uses a hierarchical authorization model. A client operation might require permissions at multiple levels. For instance, consuming from a topic requires both topic-level READ and consumer group-level READ/DESCRIBE. If any required permission is missing, the authorization fails.

After resolving these, the next error you might encounter is likely related to metadata fetching or another system component that was indirectly affected, such as LeaderNotAvailable if the authorization issue prevented a controller from re-electing a leader, or NoBrokersAvailable if the client couldn’t even establish a connection due to initial security misconfigurations.