Grafana’s templating engine is failing to resolve a variable used in a panel’s query, leading to the panel displaying an error instead of data. This usually means the variable isn’t being set correctly or the data source for the variable is unavailable.

Common Causes and Fixes:

  1. Typo in Variable Name (Most Common)

    • Diagnosis: Double-check the variable name in the panel’s query against the defined variable name in the dashboard’s "Variables" settings. Even a single character difference, case sensitivity, or a leading/trailing space will break it.
    • Fix: In the panel’s query editor, ensure the variable is referenced correctly, e.g., ${my_variable}. In the dashboard’s "Variables" settings, verify the "Name" field matches exactly, e.g., my_variable.
    • Why it works: The templating engine performs exact string matching. Any discrepancy prevents the variable from being substituted.
  2. Variable Data Source Unavailable or Incorrect

    • Diagnosis: Go to the dashboard’s "Variables" settings. For the problematic variable, check its "Data source" setting. Ensure this data source is correctly configured and accessible from Grafana. Test the data source connection if possible.
    • Fix: If the data source is wrong, select the correct one from the dropdown. If it’s inaccessible, troubleshoot the data source itself (network, authentication, service status). For example, if using Prometheus, ensure the Prometheus server is running and reachable on http://prometheus-server:9090.
    • Why it works: The variable needs to query a data source to fetch its possible values. If that source is down or misconfigured, the variable can’t populate.
  3. Variable Query Returns No Values

    • Diagnosis: In the "Variables" settings, find the variable. Below the query editor for that variable, there’s usually a "Preview of values" or similar section. If this is empty, the query isn’t returning anything. Run the query directly against the data source to see why.
    • Fix: Modify the variable’s query to return at least one value. For example, if your Prometheus query is label_values(up, instance), and there are no instance labels with up metrics, it will return empty. You might need to adjust the metric or label selector, or ensure data is being scraped. A common fix for a Prometheus query might be to ensure it’s selecting a valid metric like label_values(node_exporter_build_info, instance).
    • Why it works: If the variable has no options to choose from, it cannot be set, and subsequent panel queries that depend on it will fail.
  4. Variable is Not Selected or "Current" Value is Invalid

    • Diagnosis: Look at the dropdown at the top of the dashboard for the variable. If it’s empty or shows an error, or if the selected value for that variable doesn’t exist in the data source at that moment (e.g., a server that was running yesterday is down today), the panel will error.
    • Fix: Manually select a valid option from the variable dropdown. If there are no valid options, you need to fix the variable’s query (see point 3). For a "multi-value" variable, ensure the "Include All option" is enabled if you want to allow selecting all possible values, and that the query accounts for .* if using the "All" option.
    • Why it works: Panels often use the currently selected variable value in their queries. If that value is absent or invalid, the panel’s query becomes unresolvable.
  5. Incorrect Variable Type or Regex Filtering

    • Diagnosis: In the "Variables" settings, check the "Type" of the variable (e.g., Query, Custom, Textbox). If it’s a "Query" type, examine the "Regex" field. A poorly written regex can filter out all valid values returned by the query.
    • Fix: Adjust the regex to correctly capture the desired values. For example, if your query returns server-prod-01, server-prod-02, but your regex is /server-dev-./, it will filter everything out. A common fix for Prometheus label values might be /(.*)/ to capture everything.
    • Why it works: The regex acts as a post-processor for the variable query results. If it’s too restrictive, it will leave no options for the variable.
  6. Permissions Issue with Data Source

    • Diagnosis: The Grafana service account might not have the necessary permissions to query the data source for variable values. This is more common with secured data sources like databases or APIs.
    • Fix: Ensure the user/service account Grafana uses to connect to the data source has read access to the relevant tables or metrics. For example, if querying PostgreSQL, the Grafana user needs SELECT privileges on the table providing variable options.
    • Why it works: The variable query, like any other panel query, is subject to the underlying data source’s access control.
  7. Time Range Mismatch for Variable Data

    • Diagnosis: Some variable queries are sensitive to the dashboard’s time range. If the query is designed to fetch data from a specific period that’s outside the current dashboard’s selected time range, it might return empty.
    • Fix: In the variable settings, check if the "Refresh" option is set to "On Time Range Change". If so, ensure your variable query is compatible with varying time ranges, or set "Refresh" to "On Dashboard Load" if the variable values are static. For time-series data, ensure the query is selecting a relevant time frame or using time().
    • Why it works: If the variable query relies on data that only exists within the dashboard’s current time window, and that window is set to a period with no relevant data, the variable will be empty.

The next error you’ll likely encounter after fixing this is a panel query error related to an invalid or missing metric name, if the variable was supposed to supply that.

Want structured learning?

Take the full Grafana course →