Forward Fluent Bit Logs to a Fluentd Aggregator (2026)

Fluent Bit is surprisingly bad at sending logs to Fluentd by default.

Let’s say you’ve got Fluent Bit running on your edge machines, collecting logs from your applications, and you want to send them to a central Fluentd instance for aggregation, parsing, and storage. You’ve configured Fluent Bit with an [OUTPUT] plugin for Fluentd, but your logs are just… disappearing. They aren’t showing up in Fluentd.

This usually happens because Fluent Bit, by default, tries to send logs using the forward protocol, which Fluentd also supports, but it expects a specific configuration on the Fluentd side that most people miss. The problem isn’t that the forward protocol is broken, but that the default settings on both sides don’t align for a seamless handshake.

Here’s why it’s failing and how to fix it, covering the most common culprits:

1. Fluentd isn’t listening for `forward` protocol on the expected port.

This is the most frequent offender. Fluentd needs a specific input plugin configured to accept incoming forward protocol connections.

Diagnosis: On your Fluentd aggregator server, check its configuration file (often /etc/fluentd/fluent.conf or a file in /etc/fluentd/conf.d/). Look for a <source> block that specifies type forward. If it’s missing or configured for a different port, this is your issue.

Fix: Add or modify a <source> block in your Fluentd configuration. For example:

<source>
  @type forward
  port 24224  # Or any other port you prefer
  bind 0.0.0.0 # Listen on all interfaces
</source>

Why it works: This tells Fluentd to open a TCP socket on port 24224 and specifically listen for data formatted according to the forward protocol, which Fluent Bit will be sending. bind 0.0.0.0 is crucial if your Fluent Bit instances are not on the same machine as Fluentd.

2. Fluent Bit is configured to use the wrong port or host for Fluentd.

Simple, but easy to overlook. Your Fluent Bit output configuration must correctly point to your Fluentd aggregator.

Diagnosis: Examine your Fluent Bit configuration file (often /etc/fluent-bit/fluent-bit.conf or /etc/fluentbit/fluent-bit.conf). Locate the [OUTPUT] section for Fluentd.

Fix: Ensure the Host and Port parameters in your Fluent Bit output plugin match the <source> configuration in Fluentd.

[OUTPUT]
    Name          forward
    Match         *
    Host          your_fluentd_aggregator_ip_or_hostname
    Port          24224 # Must match Fluentd's <source> port
    Retry_Limit   False # Or a specific number of retries

Why it works: This ensures Fluent Bit knows precisely where to send its log data. If the host is wrong, it won’t even attempt to connect. If the port is wrong, it will connect to the wrong service or nothing at all.

3. Network connectivity issues or firewall blocking.

Even if configurations are perfect, the network itself can be the blocker.

Diagnosis: From the machine running Fluent Bit, try to telnet or nc to the Fluentd aggregator on the configured port: telnet your_fluentd_aggregator_ip_or_hostname 24224. If it times out or says "connection refused," the network path is blocked.
Fix:
- Firewall: Ensure that the firewall on the Fluentd aggregator server (e.g., ufw, firewalld, iptables) allows incoming TCP connections on port 24224 from your Fluent Bit source IPs.
  - For ufw: sudo ufw allow 24224/tcp
  - For firewalld: sudo firewall-cmd --zone=public --add-port=24224/tcp --permanent && sudo firewall-cmd --reload
- Network Routing: Verify that the Fluent Bit machines can route traffic to the Fluentd aggregator’s IP address.
Why it works: This opens the communication channel, allowing Fluent Bit’s outgoing packets to reach Fluentd’s listening socket.

4. Fluentd’s `forward` input plugin is misconfigured with `shared_key` or `buffer`.

When Fluentd’s forward input plugin is configured with specific buffer settings or a shared_key for message signing, it can cause handshake failures if not matched by Fluent Bit.

Diagnosis: Check the <source> block for the forward type in Fluentd. Look for parameters like shared_key, buffer_mode, buffer_chunk_limit, etc.
Fix:
- Remove shared_key: If you don’t need message authentication, remove shared_key from the Fluentd <source> configuration.
- Match buffer settings: If you do need shared_key, ensure Fluent Bit’s output plugin is also configured with the exact same shared_key. For buffer settings, typically the defaults are fine unless you have specific buffering needs.
- Fluent Bit forward output:
```
[OUTPUT]
    Name          forward
    Match         *
    Host          your_fluentd_aggregator_ip_or_hostname
    Port          24224
    Shared_Key    your_secret_key # Only if Fluentd <source> has it
```
Why it works: The forward protocol has optional security features. If Fluentd expects a signed message (via shared_key) and Fluent Bit doesn’t provide it, the connection will be rejected. Mismatched buffer settings can also lead to malformed data that Fluentd cannot process.

5. Fluent Bit’s `buffer.chunk_size` is too large for Fluentd’s processing.

Fluent Bit buffers logs before sending them. If these chunks are excessively large, Fluentd might struggle to process them, especially if it also has buffering or parsing limitations.

Diagnosis: Check Fluent Bit’s [SERVICE] section for Log_Buffer_Size or Log_Flush settings. Also, check the [OUTPUT] section for buffer.chunk_size or similar.

Fix: Reduce the buffer.chunk_size in your Fluent Bit output configuration. A common starting point is 1m (1 megabyte).

[OUTPUT]
    Name          forward
    Match         *
    Host          your_fluentd_aggregator_ip_or_hostname
    Port          24224
    Buffer.Chunk_Size 1m # Reduce chunk size
    Buffer.Max_Chunks 10 # Adjust max chunks if needed

Why it works: Smaller chunks are easier for Fluentd to ingest and parse without timing out or running into internal buffer limits. This provides a more granular flow of data.

6. Fluentd is running out of resources (CPU, Memory, File Descriptors).

Your Fluentd aggregator might be overwhelmed, unable to accept new connections or process incoming data efficiently.

Diagnosis: Monitor your Fluentd server’s resource utilization (top, htop, free -m). Check Fluentd’s logs for any errors related to OOM (Out Of Memory), disk space, or too many open files. You can check open file limits with ulimit -n.
Fix:
- Resource Allocation: Increase CPU and RAM allocated to the Fluentd process or the server it’s running on.
- File Descriptors: Increase the maximum number of open file descriptors for the Fluentd user. Edit /etc/security/limits.conf and add/modify:
```
* soft nofile 65536
* hard nofile 65536
root soft nofile 65536
root hard nofile 65536
```
  (You’ll need to restart Fluentd, or even the server, for this to take full effect).
- Fluentd Configuration: Optimize Fluentd’s buffering and parsing plugins to reduce resource consumption.
Why it works: Ensuring Fluentd has sufficient resources allows it to maintain its listening sockets, process incoming data streams, and write them to storage without crashing or becoming unresponsive.

Once you’ve addressed these, your Fluent Bit logs should start flowing into Fluentd. The next thing you’ll likely encounter is a flood of poorly parsed JSON in Fluentd, leading you to the topic of Fluentd parsing plugins.

1. Fluentd isn’t listening for forward protocol on the expected port.