Articles

Fix Grafana "Alertmanager Not Reachable" Configuration Errors (2026)

April 18, 2026 5 min read

Grafana’s Alertmanager is failing to report its status, indicating a breakdown in communication between Grafana itself and the Alertmanager service. This isn’t just a UI glitch; it means your alerts might not be firing, or worse, you’re not getting any feedback on the health of your alerting system.

Common Causes and Fixes for "Alertmanager Not Reachable"

1. Incorrect Alertmanager URL in Grafana Configuration

  • Diagnosis: Check the alerting section of your Grafana configuration file (usually grafana.ini or a file in /etc/grafana/conf.d/). Look for the [alerting] block and the [alerting.alertmanager] sub-block. Specifically, verify the url parameter.
  • Common Mistake: The URL might be missing the scheme (http:// or https://), use an incorrect port (default is 9093), or point to the wrong hostname/IP.
  • Example Configuration:
    [alerting]
enabled = true

[alerting.alertmanager]
enabled = true
url = http://localhost:9093
  • Fix: Ensure the url is correctly formatted and points to the actual address where your Alertmanager is listening. If Alertmanager is running on the same host as Grafana, http://localhost:9093 is common. If it’s on a different machine, use its IP or hostname.
  • Why it Works: Grafana directly probes this URL to determine Alertmanager’s health. An incorrect URL means Grafana is looking in the wrong place, hence "not reachable."

2. Network Connectivity Issues Between Grafana and Alertmanager

  • Diagnosis: From the server where Grafana is running, try to curl the Alertmanager URL you’ve configured.
    curl -v http://localhost:9093/api/v1/status
    (Replace http://localhost:9093 with your actual Alertmanager URL).
  • Common Mistake: Firewalls blocking the port, incorrect routing, or Alertmanager not being exposed to the network Grafana is on. If Grafana and Alertmanager are in different Docker networks or Kubernetes namespaces, this is a prime suspect.
  • Fix:
    • Firewall: Ensure the Alertmanager port (default 9093) is open on the Alertmanager host’s firewall. For ufw on Ubuntu: sudo ufw allow 9093/tcp.
    • Docker: If using Docker, ensure Grafana and Alertmanager are on the same custom network, or that the Alertmanager port is published to the host accessible by Grafana. For example, docker run -p 9093:9093 ... prom/alertmanager.
    • Kubernetes: Ensure Grafana’s Service can reach Alertmanager’s Service, potentially by using FQDNs like alertmanager.monitoring.svc.cluster.local:9093.
  • Why it Works: curl directly tests the network path and service availability. If curl fails, Grafana won’t be able to connect either.

3. Alertmanager Service Not Running or Crashed

  • Diagnosis: Check the status of the Alertmanager service.
    • Systemd: sudo systemctl status alertmanager
    • Docker: docker ps | grep alertmanager
    • Kubernetes: kubectl get pods -l app=alertmanager -n <your-namespace>
  • Common Mistake: The Alertmanager process might have exited due to an error, insufficient resources, or a configuration problem within Alertmanager itself.
  • Fix:
    • Systemd: sudo systemctl start alertmanager and sudo systemctl enable alertmanager (to start on boot).
    • Docker: docker start <alertmanager_container_id> or docker-compose up -d.
    • Kubernetes: Investigate pod logs (kubectl logs <alertmanager-pod-name> -n <your-namespace>) for errors and restart the deployment if necessary.
  • Why it Works: Grafana can only reach an Alertmanager that is actively running and listening on its configured port.

4. Incorrect Alertmanager Configuration (Alertmanager’s side)

  • Diagnosis: While Grafana reports "not reachable," sometimes the issue is that Alertmanager isn’t properly configured to receive requests or is in a bad state. Check Alertmanager’s own logs for errors.
  • Common Mistake: Alertmanager might be configured to listen on a different interface than expected (e.g., 127.0.0.1 instead of 0.0.0.0), or it might have failed to load its configuration and is in an unhealthy state.
  • Fix: Review Alertmanager’s configuration file (alertmanager.yml) and its startup logs for any errors. Ensure the listen_address in Alertmanager’s configuration is set to 0.0.0.0:9093 or the specific IP Grafana can reach. Restart Alertmanager after correcting its configuration.
  • Why it Works: If Alertmanager isn’t listening on the correct network interface or failed to start properly, Grafana will be unable to establish a connection.

5. TLS/SSL Certificate Issues (if using HTTPS)

  • Diagnosis: If your Alertmanager URL starts with https://, Grafana needs to trust its certificate. Check Grafana’s logs for TLS handshake errors.
  • Common Mistake: The certificate used by Alertmanager is self-signed and not trusted by Grafana, the certificate has expired, or the hostname in the certificate doesn’t match the URL Grafana is using.
  • Fix:
    • Trusted CA: Use a certificate signed by a Certificate Authority (CA) that Grafana trusts.
    • Custom CA: If using a custom CA, configure Grafana to trust it by setting [certs_dir] in grafana.ini to a directory containing the CA certificate, or by specifying ca_cert_file in the Alertmanager configuration within grafana.ini (if Grafana supports it directly for its Alertmanager connection, otherwise it’s a system trust store issue).
    • Insecure Skip Verify (Not Recommended for Production): For testing, you could disable TLS verification in Grafana’s Alertmanager config if Alertmanager is configured with server_name and client_certs (though direct insecure_skip_verify is not a common direct option for Alertmanager URL in Grafana’s alerting.alertmanager block). A more robust approach is fixing the certificates.
  • Why it Works: Secure connections require both ends to authenticate and trust each other. Certificate mismatches or untrusted CAs break this trust.

6. Grafana Internal State/Cache Issues

  • Diagnosis: Sometimes Grafana’s internal state can get confused. This is less common but can happen.
  • Common Mistake: Grafana might be holding onto stale connection information.
  • Fix: Restart the Grafana service.
    • Systemd: sudo systemctl restart grafana-server
    • Docker: docker restart <grafana_container_id>
    • Kubernetes: kubectl rollout restart deployment <grafana-deployment-name> -n <your-namespace>
  • Why it Works: Restarting Grafana forces it to re-read its configuration and re-establish connections, clearing any transient internal states.

After resolving these, the next error you might encounter is related to Alertmanager not being able to route alerts correctly, often manifesting as "No receivers configured" or specific routing rule failures.

Want structured learning?

Take the full Grafana course →