Fluentd’s forward input and output plugins can secure their communication using mutual TLS, ensuring both the client and server authenticate each other before exchanging logs.

Here’s Fluentd forwarding logs between two instances, fluentd-server and fluentd-client, with mutual TLS enabled.

First, let’s set up our certificates. We’ll need a Certificate Authority (CA) to sign both the server and client certificates.

# Create CA private key
openssl genrsa -out ca.key 2048

# Create CA certificate
openssl req -x509 -new -nodes -key ca.key -sha256 -days 1825 -out ca.crt \
  -subj "/CN=MyFluentdCA"

# Create server private key
openssl genrsa -out server.key 2048

# Create server certificate signing request (CSR)
openssl req -new -key server.key -out server.csr \
  -subj "/CN=fluentd-server.example.com"

# Sign server CSR with CA
openssl x509 -req -in server.csr -CA ca.crt -CAkey ca.key -CAcreateserial \
  -out server.crt -days 365 -sha256

# Create client private key
openssl genrsa -out client.key 2048

# Create client certificate signing request (CSR)
openssl req -new -key client.key -out client.csr \
  -subj "/CN=fluentd-client.example.com"

# Sign client CSR with CA
openssl x509 -req -in client.csr -CA ca.crt -CAkey ca.key -CAcreateserial \
  -out client.crt -days 365 -sha256

Now, we need to combine the certificate and private key for each side.

On fluentd-server:

cat server.crt server.key > server.pem

On fluentd-client:

cat client.crt client.key > client.pem

Ensure ca.crt is available on both fluentd-server and fluentd-client.

Fluentd Server Configuration (fluentd-server.conf)

The server listens for incoming connections on a specific port, configured to require client authentication.

<source>
  @type forward
  port 24224
  @id input_forward
  <security>
    shared_key "changeme" # This is for basic auth if mTLS is NOT used.
                           # For mTLS, it's not strictly necessary but good to have
                           # as a fallback or for non-mTLS clients.
  </security>
  <transport tls>
    cert_each false
    cert /etc/fluentd/certs/server.pem
    private_key /etc/fluentd/certs/server.pem
    ca_cert /etc/fluentd/certs/ca.crt
    verify_mode 1 # 1 means verify client certificate
  </transport>
</source>

<match **>
  @type stdout
  @id output_stdout
</match>
  • cert: Path to the server’s combined certificate and private key.
  • private_key: Same as cert when using a combined file.
  • ca_cert: Path to the CA certificate used to verify client certificates.
  • verify_mode 1: This is crucial. It tells the server to request a client certificate and verify it against the provided ca_cert.

Fluentd Client Configuration (fluentd-client.conf)

The client connects to the server, presenting its own certificate for authentication.

<source>
  @type dummy
  dummy {"message":"Hello from client!"}
  @id input_dummy
  rate 1
</source>

<match **>
  @type forward
  host fluentd-server.example.com
  port 24224
  @id output_forward
  <security>
    shared_key "changeme" # Same as server, for fallback/non-mTLS.
  </security>
  <transport tls>
    cert /etc/fluentd/certs/client.pem
    private_key /etc/fluentd/certs/client.pem
    ca_cert /etc/fluentd/certs/ca.crt
    insecure_skip_verify false # Ensure we are verifying the server cert
  </transport>
</match>
  • cert: Path to the client’s combined certificate and private key.
  • private_key: Same as cert for a combined file.
  • ca_cert: Path to the CA certificate used to verify the server’s certificate.
  • insecure_skip_verify false: This ensures the client verifies the server’s certificate against the provided ca_cert.

Running Fluentd

  1. On fluentd-server: Place server.pem and ca.crt in /etc/fluentd/certs/. Start Fluentd: fluentd -c fluentd-server.conf

  2. On fluentd-client: Place client.pem and ca.crt in /etc/fluentd/certs/. Start Fluentd: fluentd -c fluentd-client.conf

You should see the "Hello from client!" message appear in the fluentd-server’s stdout.

The verify_mode parameter on the server is what enforces mutual TLS. When set to 1, the server will not only present its certificate but also request one from the client. If the client doesn’t provide a valid certificate (signed by the same CA), the connection will be rejected. Conversely, on the client, insecure_skip_verify false ensures it trusts the server’s identity.

After successfully configuring mutual TLS, the next common hurdle is ensuring proper certificate rotation and management, especially in production environments where certificates have expiration dates.

Want structured learning?

Take the full Fluentd course →