Add, Remove, and Rename Fields in Fluent Bit with Record Modifier (2026)

Record Modifier is Fluent Bit’s Swiss Army knife for manipulating event records.

Let’s see it in action. Imagine we’re getting logs from a web server, and we want to add a region field and remove a noisy user_agent field, then rename status_code to http_status.

Here’s a simplified Fluent Bit configuration:

[SERVICE]
    Flush        1
    Daemon       off
    Log_Level    info

[INPUT]
    Name         tail
    Path         /var/log/nginx/access.log
    Tag          web.access

[FILTER]
    Name         record_modifier
    Match        web.access
    # Add a new field 'region'
    Set          region "us-east-1"
    # Remove the 'user_agent' field
    Remove       user_agent
    # Rename 'status_code' to 'http_status'
    Rename       status_code http_status

[OUTPUT]
    Name         stdout
    Match        web.access
    Format       json

If our access.log contains a line like this:

192.168.1.1 - - [10/Oct/2023:13:55:36 +0000] "GET /index.html HTTP/1.1" 200 1234 "http://referrer.com" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36"

And we’ve configured Fluent Bit to parse it into fields like remote, user, time, method, uri, status_code, bytes, referer, and user_agent, the output to stdout will look like this:

{
    "web.access": {
        "region": "us-east-1",
        "remote": "192.168.1.1",
        "user": "-",
        "time": "10/Oct/2023:13:55:36 +0000",
        "method": "GET",
        "uri": "/index.html",
        "http_status": 200,
        "bytes": 1234,
        "referer": "http://referrer.com"
    }
}

Notice how region is added, user_agent is gone, and status_code is now http_status.

Record Modifier operates on the key-value pairs within an event record. It’s part of the filter plugin chain, meaning it processes records after they’ve been ingested by an input plugin and before they’re sent to an output plugin. The Match directive, just like in other Fluent Bit plugins, determines which records this filter will apply to.

The core commands are Set, Add, Remove, and Rename.

• Set: This is for assigning a fixed value to a field. If the field doesn’t exist, it’s created. If it exists, its value is overwritten. In our example, Set region "us-east-1" adds a new field named region with the literal string us-east-1 to every matching record.

• Add: This is similar to Set but specifically for adding a field only if it doesn’t already exist. It won’t overwrite an existing field. This is useful for ensuring a field is present without accidentally deleting existing data. For instance, Add source_ip $remote would add the value of the remote field to a new field called source_ip if source_ip wasn’t already defined.

• Remove: This simply deletes a field from the record. Remove user_agent in our example ensures that the potentially verbose user_agent string is discarded, cleaning up the log data.

• Rename: This changes the name of an existing field. Rename status_code http_status is a straightforward renaming operation. The original field status_code is removed, and a new field http_status is created with the original value.

The power of Record Modifier comes from its ability to perform these operations directly within Fluent Bit’s processing pipeline, reducing the need for external scripting or complex transformations in downstream systems. You can chain multiple Set, Remove, and Rename operations within a single record_modifier filter block, allowing for sophisticated data grooming.

One subtle but powerful aspect is that Set and Add can use values from other fields within the same record. This allows for dynamic field manipulation. For example, if you wanted to create a new field that concatenates the method and URI, you could use Set request "$method $uri". The $ prefix tells Record Modifier to interpolate the value of the specified field. This capability extends to more complex scenarios, such as building a full request path string or deriving new metrics based on existing ones.

After you’ve successfully added, removed, and renamed fields, the next common task is to parse more complex nested data within your logs, often using the lua filter.