The most surprising thing about shipping Fly.io application logs externally is that you often don’t need to change your application code at all.
Here’s a standard fly.toml configuration for an application that uses the default stdout logging:
app = "my-app"
primary_region = "ord"
[build]
image = "ghcr.io/my-org/my-app:latest"
[http_service]
internal_port = 8080
force_https = true
auto_stop_machines = true
auto_rollback_machines = true
[log_defaults]
level = "info"
When your app runs, its stdout and stderr streams are captured by Fly.io’s infrastructure and made available via fly logs. But if you want those logs to go to a dedicated logging service like Datadog, Splunk, or even a simple cloud object storage bucket, you’ll configure a log_destination.
Let’s say you want to send logs to a Syslog endpoint. You’d add a log_destination block to your fly.toml:
[[log_destinations]]
type = "syslog"
address = "syslog.example.com:514"
protocol = "udp"
hostname = "my-app-prod-01" # Optional: custom hostname in log messages
app_name = "my-app" # Optional: custom app name in log messages
With this in place, Fly.io’s agent on the machine running your app will intercept the stdout/stderr and forward it to syslog.example.com over UDP. Your application itself doesn’t know or care; it just prints to standard output.
Here’s how it looks in action. Imagine your app logs a request like this:
2023-10-27T10:00:00.123Z INFO request_id="abcde12345" method=GET path=/users user_id=9876
If your log_destination is configured for Syslog, Fly.io’s agent will package this message and send it. The exact format depends on the Syslog protocol and your specific configuration, but it might look something like this (simplified):
<13>Oct 27 10:00:00 my-app-prod-01 my-app[123]: 2023-10-27T10:00:00.123Z INFO request_id="abcde12345" method=GET path=/users user_id=9876
The <13> is the Syslog priority and facility code, the timestamp, the hostname you specified, the app name, and then your original log line.
There are several type options for log_destination:
syslog: For sending logs to a Syslog server. You specifyaddressandprotocol(udportcp).journald: For sending logs to the local systemd journal. This is useful if you’re running other services on the same machine that consume the journal.http: For sending logs to an HTTP endpoint (like a webhook). You specifyurl.stdout: This is the default. It means logs go to Fly.io’s managed log aggregation.
You can also filter logs before they are sent. For example, to only send logs with a severity level of error or higher to an external destination:
[[log_destinations]]
type = "http"
url = "https://logs.example.com/ingest"
level = "error" # Only send logs at 'error' level or above
This level parameter acts as a threshold. Any log message with a level lower than the specified one (e.g., debug, info, warn when level is error) will not be sent to this destination.
When you use type = "http", the logs are sent as JSON payloads in batches. Fly.io’s agent will POST these batches to your specified url. The exact structure of the payload is documented by Fly.io, but it typically includes metadata like the app name, machine ID, and the log line itself.
One common pattern is to use stdout for general observability within Fly.io’s platform (e.g., for quick fly logs checks) and then configure a separate log_destination for critical logs or long-term archival. You can have multiple log_destination blocks, each with its own type, configuration, and level filtering.
The level filter is applied independently to each log_destination. So, you could send info level logs to a central SIEM and only error level logs to a high-cost archival storage.
If you’re sending logs via http and your endpoint is slow to respond, the Fly.io agent has an internal buffer. If that buffer fills up because the destination can’t keep up, the agent will start dropping logs to prevent overwhelming the machine. You won’t typically see an error from Fly.io itself for this; your logs will just stop appearing at the destination.
To send logs to S3, you’d use the http type and point it to an S3 compatible endpoint, often using a service like AWS API Gateway or a custom Lambda function that writes to S3.
The next step is often to configure structured logging within your application to make those external logs more useful.