Fluent Bit can send records to multiple destinations based on pattern matching of their tags.

Let’s see it in action. Imagine you have logs from different services, and you want to send application logs to Elasticsearch, system logs to Splunk, and anything else to a local file.

Here’s a simplified fluent-bit.conf that accomplishes this:

[SERVICE]
    Flush        5
    Daemon       off
    Log_Level    info

[INPUT]
    Name        tail
    Path        /var/log/app/*.log
    Tag         app.*

[INPUT]
    Name        tail
    Path        /var/log/syslog
    Tag         syslog.*

[INPUT]
    Name        tail
    Path        /var/log/other/*.log
    Tag         other.*

[OUTPUT]
    Name        es
    Match       app.*
    Host        elasticsearch.example.com
    Port        9200
    Logstash_Format on
    Replace_Dots on

[OUTPUT]
    Name        splunk
    Match       syslog.*
    Host        splunk.example.com
    Port        8088
    Splunk_Token YOUR_SPLUNK_TOKEN
    Splunk_Source app_logs
    Splunk_Sourcetype syslog

[OUTPUT]
    Name        file
    Match       other.*
    Path        /var/log/fluent-bit/fallback.log

In this configuration:

  • The [SERVICE] section sets up basic Fluent Bit behavior.
  • We have three [INPUT] sections. The tail input reads from specified files.
    • The first tail input reads all files ending in .log within /var/log/app/ and assigns them the tag app.*.
    • The second reads /var/log/syslog and tags it syslog.*.
    • The third reads files in /var/log/other/ and tags them other.*.
  • Then, we have three [OUTPUT] sections. The crucial part is the Match directive.
    • The es output uses Match app.*. This means any record whose tag starts with app. will be sent to Elasticsearch.
    • The splunk output uses Match syslog.*. Records tagged with syslog.* will go to Splunk.
    • The file output uses Match other.*. Anything tagged with other.* will be written to /var/log/fluent-bit/fallback.log.

The Match directive supports glob-style wildcard matching. The asterisk (*) matches zero or more characters. So, app.* will match tags like app.service1, app.users, or even just app. This allows for flexible routing based on how you structure your tags. You can also use ? for a single character match, or [...] for character set matching.

The most surprising true thing about this is that the Match directive doesn’t just filter which output gets the record; it also influences how the record is processed by that output plugin. Some plugins might have specific configurations that are only enabled or altered when a particular tag pattern is matched. For example, an output might send a record to a different index in Elasticsearch if its tag matches app.production.* versus app.staging.*. This allows for granular control over data routing and storage based on the origin and context conveyed by the tag.

The next concept you’ll want to explore is how to combine multiple Match patterns within a single output.

Want structured learning?

Take the full Fluentbit course →