Grafana transformations let you manipulate query results after they’re fetched, which is surprisingly powerful and often overlooked.
Let’s see this in action. Imagine we have two Prometheus queries: one for http_requests_total and another for http_request_duration_seconds_bucket.
{
"datasource": "Prometheus",
"queries": [
{
"refId": "A",
"expr": "sum(rate(http_requests_total{job='my_app'}[5m])) by (handler)",
"format": "time_series",
"legendFormat": "Requests ({{handler}})"
},
{
"refId": "B",
"expr": "histogram_quantile(0.95, sum(rate(http_request_duration_seconds_bucket{job='my_app'}[5m])) by (le, handler))",
"format": "time_series",
"legendFormat": "95th Percentile ({{handler}})"
}
]
}
By default, these would appear as two separate panels or two distinct sets of data on a single panel. But we want to see the 95th percentile latency next to the request count for each handler.
This is where transformations shine. We’ll add a "Merge" transformation.
Transformation: Merge
- Operation: Merge
- From: A, B
- By: Field with label "handler"
When you apply this, Grafana takes the results from queries A and B and combines them based on a common field. In our case, the handler label is present in both Prometheus queries. The result is a single table where each row represents a unique handler, and you have columns for both the request count and the 95th percentile latency for that handler.
The underlying mechanism is that Grafana treats each query result as a table. The "Merge" transformation performs a relational join operation. It looks for a common key (here, the handler label, which becomes a field in Grafana’s data frame representation) and stitches together rows from different tables that share that key.
But what if the labels aren’t perfectly aligned, or you want to perform more complex combinations? That’s where "Join" comes in.
Transformation: Join
Let’s say query A gives us request counts by handler, and query B gives us error counts by handler and status_code.
{
"datasource": "Prometheus",
"queries": [
{
"refId": "A",
"expr": "sum(rate(http_requests_total{job='my_app'}[5m])) by (handler)",
"format": "time_series",
"legendFormat": "Requests ({{handler}})"
},
{
"refId": "B",
"expr": "sum(rate(http_errors_total{job='my_app'}[5m])) by (handler, status_code)",
"format": "time_series",
"legendFormat": "Errors ({{handler}}, {{status_code}})"
}
]
}
If we just "Merge" this, we might get duplicate rows or unexpected combinations because query B has an additional dimension (status_code). The "Join" transformation offers more control.
- Operation: Join
- From: A, B
- Join by: Field "handler"
- Outer join: (optional, choose "Inner", "Left", "Right", "Outer")
"Join" is more explicit about how to combine tables. You specify which fields to join on (like handler) and the type of join. An "Inner join" will only keep rows where a handler exists in both A and B. A "Left join" (keeping A as the primary) would show all handlers from A, and if a matching handler exists in B, it would add its data; otherwise, B’s columns would be null for that handler. This is crucial when one data source might have more granular detail than another, and you want to preserve the broader set.
Beyond combining data, "Reshape" is your go-to for pivoting data. Imagine you have time-series data where each series is a different metric, but you want a table where rows are timestamps and columns are metrics.
Transformation: Reshape
- Operation: Reshape
- Mode: Table
- Add time index: (checked)
- Index by: Time
Query results often come in a "long" format (many rows, few columns, e.g., timestamp, metric_name, value). "Reshape" with "Table" mode transforms this into a "wide" format (fewer rows, many columns, e.g., timestamp, metric_a_value, metric_b_value). It takes a column containing distinct identifiers (like metric names) and turns those identifiers into new columns, with the corresponding values populating those columns at each timestamp. This is indispensable for creating cross-series comparisons in table panels or when your downstream analysis expects a wide data structure.
The most surprising thing about transformations is how they blur the lines between data fetching and data presentation. You can perform complex data wrangling within Grafana, reducing the need for pre-processing at the data source or in external tools.
Consider a scenario where you have a Prometheus query returning a single value, and another query returning a time series. You want to display that single value as a constant line across the time series chart.
{
"datasource": "Prometheus",
"queries": [
{
"refId": "A",
"expr": "my_static_threshold",
"format": "time_series"
},
{
"refId": "B",
"expr": "avg_over_time(my_metric[1h])",
"format": "time_series"
}
]
}
Applying a "Merge" transformation on "Time" will align these. However, the single value from A might not repeat correctly. You’d first use a "Convert field type" transformation on query A to ensure its value is treated as a number, then potentially a "Filter data by values" to remove nulls, and then a "Merge" transformation by "Time". The "Merge" transformation, when joining by time, effectively broadcasts the non-time-series value to match the timestamps of the time-series data, allowing you to overlay static thresholds or reference points directly onto your charts.
This ability to manipulate data after it’s been retrieved, using a declarative, visual interface, democratizes data analysis. You don’t need to be a SQL or Python expert to join, filter, and reshape data; Grafana’s transformations provide these capabilities directly.
When dealing with complex data structures, especially from sources like Elasticsearch or Loki, transformations like "Extract fields" or "Organize fields" become critical. You can pull nested JSON values into top-level fields or rename/reorder fields to make the data more readable and usable for subsequent transformations or visualizations.
The next step after mastering these fundamental transformations is often exploring the "Add field from calculation" transformation, which allows for creating entirely new fields based on arithmetic or logical operations between existing ones.