Fluent Bit is fundamentally a log processor and forwarder, but it’s not designed to store logs itself. Instead, it acts as a pipeline, taking logs from various sources, transforming them, and sending them to a backend system for analysis and storage. Grafana Loki is that backend system, purpose-built for storing and querying log data, and Fluent Bit is one of its primary ingestion agents.

Let’s see Fluent Bit in action, grabbing logs from a simple Nginx container and sending them to a local Loki instance.

First, we need a fluent-bit.conf file. Here’s a minimal setup:

[SERVICE]
    Flush        5
    Daemon       off
    Log_Level    info
    Parsers_File parsers.conf

[INPUT]
    Name         tail
    Path         /var/log/nginx/access.log
    Tag          nginx.access
    Parser       nginx

[OUTPUT]
    Name         loki
    Match        nginx.access
    Host         loki
    Port         3100
    Line_Format  key_value
    Label        app=nginx
    Label        environment=production

And a parsers.conf to understand Nginx logs:

[PARSER]
    Name        nginx
    Format      regex
    Regex       ^(?<remote>[^ ]*) (?<host>[^ ]*) (?<user>[^ ]*) \[(?<time>.*?)\] "(?<method>.*?) (?<path>.*?) (?<protocol>.*?)".*?(?<status>[0-9]*) (?<bytes>[^ ]*) "(?<referer>.*?)" "(?<user_agent>.*?)"
    Time_Key    time
    Time_Format %d/%b/%Y:%H:%M:%S %z

Now, imagine we have a Loki instance running at loki:3100 and an Nginx container whose access logs are mounted at /var/log/nginx/access.log inside the Fluent Bit container. If Fluent Bit is running with this configuration, it will:

  1. Listen: The tail input plugin constantly watches /var/log/nginx/access.log.
  2. Parse: The nginx parser extracts structured fields like remote, method, path, status, and bytes from each log line.
  3. Tag: Each parsed log is given the tag nginx.access.
  4. Route: The OUTPUT section matches logs with the nginx.access tag.
  5. Send: The loki output plugin formats these logs as key-value pairs and sends them to loki:3100.
  6. Label: Crucially, it adds the labels app=nginx and environment=production to all these logs. This is how Loki indexes and allows you to filter logs.

The problem this setup solves is the disconnect between where logs are generated (like application containers) and where they are analyzed. Without Fluent Bit, you’d have to manually collect logs, potentially aggregate them, and then figure out how to send them to Loki. Fluent Bit automates this, making it a seamless part of your observability stack.

Internally, Fluent Bit uses a robust plugin architecture. Inputs, filters, and outputs are chained together. Data flows from an input plugin, can be modified by filter plugins (not shown here, but common for adding metadata, dropping fields, etc.), and is then sent to one or more output plugins. The Match directive in the output section is key; it determines which tagged data streams are sent to which output. The Label directive in the Loki output is how you attach metadata at ingestion time to your logs in Loki. These labels are indexed by Loki and are fundamental to how you’ll query your logs later.

When you configure the loki output plugin, the Line_Format key_value is a common and useful choice. It tells Fluent Bit to serialize the parsed log fields into a key=value string for each log line sent to Loki. This makes the log content itself more structured within Loki, beyond just the indexed labels. Another common format is json, which sends the entire parsed log entry as a JSON object. The choice depends on how you want to interact with the log content within Loki’s query language.

The most surprising thing about Fluent Bit’s interaction with Loki is how it handles the absence of a central, persistent log buffer within Fluent Bit itself. Unlike some traditional log shippers that might maintain large local caches, Fluent Bit’s design prioritizes low memory footprint and efficient forwarding. If Loki is temporarily unavailable, Fluent Bit will retry sending logs based on its Flush interval and internal buffering, but it doesn’t aim to be a massive, disk-backed log store. Its strength is in the reliable, real-time transit of log data, pushing it towards its destination rather than holding onto it indefinitely.

The next step you’ll likely encounter is managing log retention and data tiers within Loki itself, as Fluent Bit’s job is primarily about getting the logs to Loki.

Want structured learning?

Take the full Fluentbit course →