New Relic’s ingest costs can be surprisingly high, not because it’s inefficient, but because the sheer volume of data collected by default can overwhelm even modest usage.
Let’s see this in action. Imagine a small-to-medium web application. By default, its New Relic agent might be sending tens of thousands of metrics every minute, plus logs, traces, and browser events. Each of these data points has a cost associated with it.
{
"metricSample": {
"name": "System/CPU/User/Percent",
"value": 0.75,
"timestamp": 1678886400000,
"attributes": {
"host": "app-server-01",
"region": "us-east-1"
}
},
"log": {
"message": "User logged in successfully",
"level": "INFO",
"timestamp": 1678886405000,
"attributes": {
"user.id": "12345",
"request.id": "abcde"
}
}
}
This is a simplified representation of the JSON New Relic receives. Each metricSample and log entry contributes to your ingest bill. The problem isn’t the data itself, but the quantity and granularity of data you’re paying to store and process.
The core problem New Relic ingest cost optimization solves is aligning your monitoring investment with your actual business needs. You’re paying for data visibility. If you’re drowning in low-value data, you’re paying for noise, not signal. The system allows you to filter, sample, and transform data before it hits the New Relic platform, reducing the volume and thus the cost.
Internally, New Relic agents (like the Java or Node.js agent) are configured with a set of "exporters" and "processors." When data is generated, it passes through these processors. By default, many processors are enabled and configured to capture a wide breadth of information. The levers you control are primarily within the agent configuration files (newrelic.yml for Java, newrelic.js for Node.js, etc.) and via New Relic’s UI for things like Log Management or APM settings. You can specify which metrics to collect, how often to collect them, which logs to forward, and the sampling rate for traces.
For example, in a Java application, you might have a newrelic.yml file. To reduce metric ingest, you can explicitly disable certain metric collections. Instead of collecting every single JVM metric, you might only enable jvm.memory.used and jvm.threads.live. This is done by adding a metrics_exclude or metrics_include section.
com.newrelic.agent.config:
metrics_exclude:
- jvm.classes.loaded
- jvm.gc.time
- jvm.gc.count
This tells the agent not to send data for jvm.classes.loaded, jvm.gc.time, and jvm.gc.count. Each of these, if collected granularly, adds up. By excluding them, you’re making a conscious decision that this specific metric is not critical for your real-time troubleshooting or cost considerations.
Similarly, for logs, you can configure the agent to only send logs that meet certain criteria (e.g., level: ERROR or message: "Exception"). This is often done via log.yml or within the main newrelic.yml under log_forwarding.
com.newrelic.agent.log:
enabled: true
log_level: INFO
forwarding:
enabled: true
max_samples_stored: 10000 # Limits local buffering, not ingest cost directly but related
include_timestamp: true
include_host: true
# Example: Filter logs to only send ERROR or WARN level
message_filter: "level IN ('ERROR', 'WARN')"
This message_filter is a crucial piece of ingest cost control for logs. If you have verbose INFO logs flooding your system, filtering them out here dramatically reduces log volume and associated costs.
The most surprising thing about New Relic ingest cost is that you’re often paying for all the data your agent can collect by default, not just what you’re actively querying. This means even if you never look at a specific metric, if the agent is configured to send it, it’s being ingested and billed. The default configurations are designed for maximum observability, which is a good starting point, but rarely the most cost-effective long-term strategy.
One of the most impactful, yet often overlooked, levers for reducing APM ingest costs is controlling the sampling rate of distributed traces. By default, New Relic often samples a high percentage of transactions. You can significantly reduce trace data volume by increasing the sampling rate. For example, instead of sampling 100% of transactions, you might set it to sample only 10% or even 1%. This means fewer individual transaction traces are sent to New Relic, but you still get a representative view of your application’s performance across your services. This is typically configured within the agent’s newrelic.yml or equivalent, often under a distributed_tracing or transaction_tracing section, with a parameter like sampling_rate.
The next challenge you’ll likely face is understanding the impact of custom attributes on your ingest costs.