Syslog isn’t just a way to send logs; it’s a surprisingly robust, albeit ancient, protocol for building a distributed observer system for your entire network.
Imagine you’ve got a bunch of devices – routers, firewalls, servers, even printers – all spewing out status messages. Syslog lets them all talk to a single, central server, so you don’t have to SSH into each one individually to check what’s going on. It’s like having a single dashboard for your entire IT department’s "what’s happening right now" feed.
Let’s see it in action. We’ll set up a basic syslog server on a Linux machine and have another device (like a network switch or even another Linux box) send a message to it.
First, the syslog server. We’ll use rsyslog, which is pretty standard.
On your syslog server (let’s say its IP is 192.168.1.100), you’ll need to configure rsyslog to listen. Edit /etc/rsyslog.conf and uncomment or add these lines:
module(load="imudp")
input(type="imudp" port="514")
module(load="imtcp")
input(type="imtcp" port="514")
This tells rsyslog to listen for messages coming in over UDP and TCP on the standard syslog port, 514. You’ll likely need to restart rsyslog:
sudo systemctl restart rsyslog
Now, let’s make sure our firewall isn’t blocking this. On the syslog server, you’d run:
sudo ufw allow 514/udp
sudo ufw allow 514/tcp
If you’re using firewalld:
sudo firewall-cmd --permanent --add-port=514/udp
sudo firewall-cmd --permanent --add-port=514/tcp
sudo firewall-cmd --reload
Next, we need to tell rsyslog where to put the incoming messages. You can create a new file in /etc/rsyslog.d/ for this, say 50-remote.conf:
*.* @192.168.1.100:514
This is a basic configuration that sends all messages (facility * and severity *) from any source to our central server. The @ symbol means it will try to send via UDP. If you wanted TCP, you’d use @@.
Now, on a client device (let’s use another Linux box), we’ll configure rsyslog to send its logs. Edit /etc/rsyslog.conf on the client and add this line at the end:
*.* @192.168.1.100:514
Again, restart rsyslog on the client:
sudo systemctl restart rsyslog
To test, let’s send a simple message from the client:
logger "Hello from the client!"
If you go back to your syslog server and check /var/log/syslog (or wherever your rsyslog is configured to log), you should see a line like this:
Dec 15 10:30:00 client-hostname Oct 15 10:30:00 client-hostname logger: Hello from the client!
The "client-hostname" part is crucial – it tells you which device sent the message.
The real power of syslog comes from its facilities and severities. Facilities categorize the type of message (e.g., auth for authentication, daemon for system daemons, kern for kernel messages). Severities indicate the urgency (e.g., emerg for emergency, alert for alert, crit for critical, err for error, warning for warning, notice for notice, info for informational, debug for debug).
You can use these to filter and direct logs. For example, on the server, you could have:
auth.* /var/log/auth.log
cron.* /var/log/cron.log
daemon.* -/var/log/daemon.log
kern.* /var/log/kern.log
mail.* -/var/log/mail.log
syslog.* -/var/log/syslog.log
user.* -/var/log/user.log
uucp.* /var/log/uucp.log
*.emerg :omuslog:
*.alert :omuslog:
*.crit :omuslog:
*.err :omuslog:
*.warning :omuslog:
*.notice :omuslog:
*.info :omuslog:
*.debug :omuslog:
The - before the log file path means rsyslog will write asynchronously, which is generally better for high-volume logs as it doesn’t block the sender. :omuslog: sends messages to the system logger and also to a remote syslog server.
The real magic is in how you can combine these. You could have a rule like *.err @192.168.1.100:514 to send only error messages to a specific server for immediate attention, while sending all other messages to a different storage.
What most people don’t realize is that the syslog protocol itself is incredibly simple: a UDP or TCP packet with a timestamp, hostname, process name, and message payload. The complexity and intelligence are added by the implementations like rsyslog, syslog-ng, or nxlog. These daemons parse the incoming messages, apply filtering and routing rules defined in their configuration files, and then decide what to do with them – log them locally, forward them to another syslog server, send them to a SIEM, or even trigger an action.
The next step is typically understanding how to parse and analyze the massive amounts of data you’ll be collecting, which leads into SIEM (Security Information and Event Management) systems.