Grafana’s alert routing system is failing because it can’t find a defined contact point for a specific alert rule, meaning alerts have nowhere to go.

Common Causes and Fixes:

  1. Typo in Contact Point Name: The most frequent culprit is a simple misspelling in the alert rule’s contact_point field when referencing the contact point defined in the Alerting configuration.

    • Diagnosis: Navigate to "Alerting" -> "Contact points" in your Grafana UI. Carefully compare the contact_point name specified in your alert rule’s configuration (usually found in "Alerting" -> "Alert rules" or within a notification policy) with the exact names listed for your contact points.
    • Fix: Edit the alert rule or its associated notification policy and correct the contact_point name to match precisely. For example, if your contact point is named production-slack and your alert rule has contact_point = production-slakc, change it to production-slack.
    • Why it works: This is a direct string match. The system looks for an exact name, and a typo breaks this lookup.
  2. Contact Point Not Created or Deleted: The contact point might have been deleted after the alert rule was configured, or it was never created in the first place.

    • Diagnosis: Go to "Alerting" -> "Contact points". Verify that a contact point with the name expected by your alert rule actually exists.
    • Fix: If the contact point is missing, create a new one under "Alerting" -> "Contact points" with the correct name and integration details (e.g., Slack webhook URL, PagerDuty API key).
    • Why it works: The alert router needs an existing, configured contact point to send notifications to; if it doesn’t exist, the lookup fails.
  3. Incorrect Notification Policy Matching: The alert rule might not be routed to the notification policy that includes the intended contact point. This happens when multiple notification policies are in play.

    • Diagnosis:
      • Go to "Alerting" -> "Notification policies".
      • Identify the specific alert rule causing the error.
      • Examine the labels on your alert rule (e.g., severity: critical, team: backend).
      • Check the "Matching labels" for each of your notification policies to see which one should match your alert rule.
      • Within the matching notification policy, check which contact point is assigned.
    • Fix: Either adjust the alert rule’s labels to match an existing policy’s criteria, or modify the notification policy’s matching labels to include the alert rule. Ensure the correct contact point is assigned to that policy. For instance, if your alert rule has severity: critical and service: api, and you have a policy matching severity: critical but not service: api, you might need to add service: api to that policy’s match or create a new policy for it. Then, ensure the desired contact point is linked to that policy.
    • Why it works: Notification policies act as a router. They use label matching to determine which contact point (or set of contact points) an alert should be sent to. If the rule’s labels don’t align with any policy’s match, or if the matching policy is assigned the wrong contact point, the system can’t fulfill the routing request.
  4. Contact Point Integration Configuration Error: The contact point itself might be configured with invalid credentials or endpoint details, causing Grafana to reject it during the routing process, even if the name is correct.

    • Diagnosis: Go to "Alerting" -> "Contact points". Edit the contact point that your alert rule is trying to use. Carefully review all integration-specific settings (e.g., Slack webhook URL, PagerDuty API key, email addresses, opsgenie_api_key, opsgenie_api_url).
    • Fix: Correct any malformed URLs, missing API keys, or incorrect recipient addresses. For a Slack integration, ensure the webhook URL is valid and accessible by Grafana. For PagerDuty, verify the API key has the necessary permissions.
    • Why it works: Even if the contact point exists by name, if its underlying integration details are broken, Grafana cannot successfully establish a connection to send the alert, leading to a "not found" type of error as the system can’t validate or use the endpoint.
  5. Grafana Internal Caching or State Issues: Less common, but Grafana’s internal state might get desynchronized, leading it to believe a contact point doesn’t exist when it actually does.

    • Diagnosis: This is harder to diagnose directly. Look for other symptoms in Grafana’s logs.
    • Fix: A restart of the Grafana server process can often clear temporary state issues. sudo systemctl restart grafana-server (on systems using systemd).
    • Why it works: Restarting the Grafana server reloads its configuration and internal state, potentially resolving transient data corruption or caching problems.
  6. Permissions Issues (Grafana Enterprise/Cloud): In Grafana Enterprise or Cloud environments, user permissions might prevent the current user from seeing or editing certain contact points, leading to an apparent "not found" if the configuration was made by another user with higher privileges.

    • Diagnosis: Check your current user’s role and permissions within Grafana. If you are not an Admin, you might not have visibility into all alerting configurations.
    • Fix: Ensure the user configuring or troubleshooting the alert rule has sufficient permissions (e.g., Admin role) to view and manage all contact points and notification policies, or have an administrator grant access.
    • Why it works: Access control restricts visibility and modification capabilities. If the contact point exists but your user can’t see it due to permissions, it effectively becomes "not found" for your context.

After fixing these, you’ll likely encounter errors related to the content of the alerts themselves, such as missing labels required by your notification policies or invalid template syntax.

Want structured learning?

Take the full Grafana course →