The Istio AuthorizationPolicy is failing to match incoming requests because the principal field is being evaluated incorrectly, leading to a denial of service even when policies should permit the traffic.
Here’s a breakdown of the common causes and how to fix them:
1. Incorrect Principal Format in AuthorizationPolicy
The most frequent culprit is a mismatch between the format of the principal Istio expects and what’s actually being presented by your workload. Istio’s AuthorizationPolicy primarily uses the STRICT JWT validation mode, which means the principal field in the policy needs to precisely match the sub (subject) claim within the JWT issued by your authentication provider.
-
Diagnosis: Examine the JWT being sent with your request. Tools like
curlwith thejwt-clior browser developer tools can help inspect thesubclaim. Compare this exact string to theprincipalfield in yourAuthorizationPolicy. -
Fix: Ensure the
principalin yourAuthorizationPolicyis identical to thesubclaim in the JWT. For example, if your JWT has{"sub": "user@example.com"}, your policy should look like this:apiVersion: security.istio.io/v1beta1 kind: AuthorizationPolicy metadata: name: my-app-auth namespace: default spec: selector: matchLabels: app: my-app action: ALLOW rules: - from: - source: principals: ["user@example.com"] -
Why it works: The
principalfield acts as a direct filter against thesubclaim in the validated JWT. Any deviation, including whitespace or casing, will result in a non-match.
2. Missing or Incorrect JWT Issuance from the Authentication Service
Your authentication service might not be issuing JWTs at all, or it could be omitting the sub claim, or even signing them with an incorrect key.
- Diagnosis: Verify that your authentication service is correctly configured to issue JWTs containing the
subclaim for authenticated users. Check the logs of your authentication service and inspect the tokens it generates. - Fix: Reconfigure your authentication service to include the
subclaim in all issued JWTs. Ensure the signing key used by the authentication service matches the public key configured in Istio’sjwtValidationconfiguration. - Why it works: Istio relies on a valid JWT with a recognizable
subclaim to perform principal-based authorization. Without it, no policy can ever match.
3. Incorrect jwtValidation Configuration in Istio
Istio needs to know how to validate the JWTs. This involves specifying the JWKS (JSON Web Key Set) URI from which to fetch public keys and potentially the issuer URL.
-
Diagnosis: Check your Istio
meshConfigfor thejwtValidationsection. Ensure thejwksUripoints to the correct endpoint of your identity provider and that theissuerfield (if used) matches the issuer in your JWTs. -
Fix: Update your
meshConfigwith the correct JWKS URI and issuer. For example:apiVersion: install.istio.io/v1alpha1 kind: IstioOperator spec: meshConfig: accessLogFile: /dev.stdout defaultConfig: gatewayTrafficPolicies: tlsSettings: mode: ISTIO_MUTUAL tracing: sampling: 100.0 jwtValidation: jwksUri: "https://your-auth-provider.com/.well-known/jwks.json" issuer: "https://your-auth-provider.com"After modifying
meshConfig, you’ll need to restart your Istio control plane components (istiod). -
Why it works: Istio uses the JWKS URI to fetch the public keys necessary to verify the signature of incoming JWTs. If this is misconfigured, Istio cannot trust the token and will reject it, preventing any authorization policy from being evaluated.
4. AuthorizationPolicy Selectors Not Matching the Target Workload
The selector in your AuthorizationPolicy might not be correctly identifying the workload you intend to protect.
-
Diagnosis: Use
kubectl get pods -o wide --show-labelsin the namespace of your workload. Compare the labels of your target pod(s) with thematchLabelsin yourAuthorizationPolicy. -
Fix: Adjust the
matchLabelsin yourAuthorizationPolicyto accurately reflect the labels on your target pod. For instance, if your pod hasapp: my-backendandversion: v1, your selector should be:spec: selector: matchLabels: app: my-backend version: v1 -
Why it works: The
selectoris how Istio identifies which workloads anAuthorizationPolicyapplies to. If the selector doesn’t match the pod’s labels, the policy is never considered for requests to that pod.
5. Incorrect principal Type or Missing requestPrincipals in AuthorizationPolicy
While less common, you might be specifying principals in a way that Istio doesn’t expect for the given context, or perhaps you’re trying to match a request principal that isn’t being generated.
- Diagnosis: Review the
fromsection of yourAuthorizationPolicy. If you’re expecting a JWT-based principal, ensure you’re usingsource.principals. If you’re dealing with mTLS, you’d usesource.principalswith SPIFFE IDs. - Fix: For JWT-based authorization, ensure your policy uses
source.principalsand that the value matches the JWTsubclaim. If you’re expecting a peer principal from mTLS, ensure yourAuthorizationPolicyis correctly configured for mTLS and that thesource.principalsfield uses the SPIFFE identity of the client (e.g.,cluster.local/ns/default/sa/my-client-sa). - Why it works:
AuthorizationPolicyhas different ways to specify principals (e.g., JWT principals, mTLS peer principals). Using the wrong type or expecting a principal that isn’t being generated by the network fabric (Istio mTLS or JWT injection) will lead to non-matches.
6. requestPrincipals Claim Mismatch (e.g., aud or iss)
Sometimes, the principal in the AuthorizationPolicy might be correct, but Istio is configured to match against a different JWT claim, or the JWT itself doesn’t contain the expected iss (issuer) or aud (audience) claims that Istio is configured to validate.
- Diagnosis: Check your
AuthorizationPolicyfor anyrequestPrincipalsfields. If present, they define which JWT claims Istio should use for principal matching. Also, inspect the JWT forissandaudclaims. - Fix: Ensure that if you’re using
requestPrincipalsin your policy, the specified claims exist in your JWT and that the values within those claims match what’s expected by the policy. If Istio is configured to expect a specificissueroraudienceinmeshConfig.jwtValidationor within theAuthorizationPolicyitself, make sure your JWT conforms. For example, if your policy hasrequestPrincipals: ["aud"], and your JWT has{"aud": "my-api-audience"}, then theprincipalin the policy should matchmy-api-audience. - Why it works: By default, Istio matches
source.principalsagainst thesubclaim. However, you can configure it to use other claims likeissoraudviarequestPrincipals. A mismatch here means Istio is looking in the wrong place within the JWT.
If you’ve addressed all these points and are still seeing issues, the next problem you’ll likely encounter is Istio’s EnvoyFilter not being applied correctly, leading to requests bypassing the authorization checks entirely.