Validate Fluentd Configuration with Dry-Run Before Deploying (2026)

Fluentd’s fluent-plugin-td isn’t just a plugin; it’s a gateway, and its configuration is the lock. You can test that lock before you ever hand over the keys.

Let’s see Fluentd’s configuration in action. Imagine you’re collecting logs from Nginx, filtering them, and sending them to a specific output.

Here’s a typical fluent.conf:

<source>
  @type tail
  path /var/log/nginx/access.log
  pos_file /var/log/td-agent/nginx-access.log.pos
  tag nginx.access
  <parse>
    @type nginx
  </parse>
</source>

<filter nginx.access>
  @type grep
  regex_type posix-regexp
  <regexp>
    key http_method
    regexp ^GET$
  </regexp>
</filter>

<match nginx.access>
  @type stdout
</match>

This configuration tells Fluentd to:

Source: Watch /var/log/nginx/access.log for new lines. It uses the nginx parser to understand the log format and tags these records with nginx.access. The pos_file keeps track of where it left off, so it doesn’t re-read old logs.
Filter: For any record tagged nginx.access, apply a grep filter. It keeps only the records where the http_method field matches the regular expression ^GET$.
Match: Send any record that makes it through the filter (i.e., GET requests) to the stdout output, meaning they’ll be printed to Fluentd’s console.

The mental model here is a pipeline. Data flows from sources, through filters, and finally to matches. Each component does one thing: ingest, transform, or output. Fluentd’s power comes from chaining these components together. The @type directive is the key, specifying which plugin to use for a given stage. The <tag> directive acts like a routing mechanism, directing data between components.

The real magic is in how Fluentd handles the data flow. It’s an event-driven system. When a new log line arrives at the tail source, it’s parsed and tagged. This tagged event then travels through the configured pipeline. Filters inspect and potentially modify the event. Matches receive the event and decide what to do with it – send it to a file, a database, a message queue, or just print it.

Now, imagine you’ve made a change to this fluent.conf and you’re about to restart your td-agent service. You don’t want to risk bringing down your log collection if there’s a typo or a logic error. This is where the dry-run comes in.

You can use Fluentd’s built-in fluentd -c /path/to/your/fluent.conf --dry-run command. This command will parse your configuration file and report any syntax errors or invalid plugin configurations. It doesn’t actually start the Fluentd process or ingest any data, but it simulates the configuration loading phase.

For example, if you accidentally mistyped @type as @typo in your fluent.conf:

<source>
  @type tail
  path /var/log/nginx/access.log
  pos_file /var/log/td-agent/nginx-access.log.pos
  tag nginx.access
  <parse>
    @typo nginx # Incorrect directive
  </parse>
</source>

Running fluentd -c /etc/td-agent/td-agent.conf --dry-run would immediately show an error like:

2023-10-27 10:30:00 +0000 [error]: Unknown directive 'typo' in /etc/td-agent/td-agent.conf:7

This tells you exactly where the problem is and what the issue is: an unknown directive. The fix is straightforward: change @typo back to @type.

What most people don’t realize is that --dry-run also validates the existence of specified plugins. If you’ve configured a plugin that isn’t installed in your Fluentd environment (e.g., you forgot to run fluent-gem install fluent-plugin-elasticsearch), --dry-run will catch it. For instance, if you had a match section like this:

<match nginx.access>
  @type elasticsearch
  host localhost
  port 9200
  logstash_format true
  logstash_prefix fluentd
  include_tag_key true
  tag_key log_tag
  flush_interval 5s
</match>

And the fluent-plugin-elasticsearch gem was not installed, fluentd --dry-run would output something along these lines:

2023-10-27 10:35:00 +0000 [error]: plugin 'elasticsearch' not found. Did you install it?

This is incredibly useful for ensuring your environment is correctly set up before you try to send data. The fix, in this case, would be to install the missing plugin: td-agent-gem install fluent-plugin-elasticsearch (or fluent-gem install fluent-plugin-elasticsearch depending on your setup).

The next step after validating your configuration is to understand how to monitor the health of your running Fluentd process.