New Relic Workflows can feel like a black box for routing alerts, but they’re actually a surprisingly simple, event-driven system that mirrors how you’d manually sort emails.

Let’s see how this works with a concrete example. Imagine you have a critical service, "AuthService," and you want any PagerDuty alerts for it to go to the "Identity" team.

Here’s the setup in New Relic:

First, you need to define a "Destination" for your alerts. This is where the alert notification will actually be sent. For PagerDuty, you’d create a destination like this:

# newrelic_notification_destination.yaml
resource "newrelic_notification_destination" "pagerduty_identity" {
  name = "PagerDuty - Identity Team"
  type = "PAGERDUTY"
  config {
    account_id = "YOUR_PAGERDUTY_ACCOUNT_ID"
    routing_key = "YOUR_PAGERDUTY_ROUTING_KEY_FOR_IDENTITY_TEAM"
  }
}
  • name: A human-readable name for this destination.
  • type: The service you’re sending notifications to (e.g., PAGERDUTY, SLACK, WEBHOOK).
  • config: Specific credentials and identifiers for the chosen service. This is where you’d put your PagerDuty account ID and the specific routing key that directs alerts to the "Identity" team’s PagerDuty service.

Next, you create the "Workflow" itself. This is the core logic that decides when and where to send notifications.

# newrelic_workflow.yaml
resource "newrelic_workflow" "auth_service_critical_alerts" {
  name        = "AuthService Critical Alerts"
  description = "Routes critical alerts for AuthService to the Identity team via PagerDuty."
  enables     = true

  filter {
    # This is the "event" that triggers the workflow
    event_type = "INCIDENT"
    attributes {
      # We only care about incidents that are "OPEN"
      "incident.state" = "OPEN"
      # And specifically for our "AuthService"
      "labels.service" = "AuthService"
      # And only "CRITICAL" severity
      "incident.severity" = "CRITICAL"
    }
  }

  destination_ids = [
    newrelic_notification_destination.pagerduty_identity.id
  ]
}
  • name and description: Self-explanatory.
  • enables: Set to true to activate the workflow.
  • filter: This is the heart of the workflow.
    • event_type: We’re listening for INCIDENT events. New Relic generates various event types (like ALERT, INCIDENT, WORKLOAD, etc.) that workflows can act upon.
    • attributes: This is where you define the conditions for triggering the workflow.
      • "incident.state" = "OPEN": We only want to act when an incident is newly opened.
      • "labels.service" = "AuthService": This is crucial for targeting. It means the alert must have a service label set to AuthService. New Relic alert conditions allow you to define labels, which are key-value pairs that attach metadata to your alerts.
      • "incident.severity" = "CRITICAL": We’re only interested in the most severe alerts.
  • destination_ids: A list of the IDs of the newrelic_notification_destination resources you want to use. In this case, it’s our pagerduty_identity destination.

When an alert fires in New Relic that matches the filter criteria (an OPEN INCIDENT for AuthService with CRITICAL severity), New Relic will send a notification to the PagerDuty - Identity Team destination.

The mental model here is simple: Event -> Filter -> Action. New Relic is constantly watching for events. When an event occurs, it checks if it matches any active workflow’s filter. If it does, it executes the destination_ids associated with that workflow.

Think of labels as the most powerful way to slice and dice your alerts. You can add labels like team, service, environment, region, component, etc., to your alert conditions. These labels are then available as attributes in your workflow filters, allowing for incredibly granular routing.

Most people don’t realize that the event_type in the filter can be more than just INCIDENT. You can also filter on ALERT events (which are the raw alert signals before they’re grouped into incidents), or even custom event types if you’re sending application logs or other data into New Relic.

The next step is to start routing alerts to multiple teams based on different conditions, perhaps using Slack for warnings and PagerDuty for criticals.

Want structured learning?

Take the full Newrelic course →