Packet marking lets you steer traffic onto different paths through your network, not just based on destination IP, but on any criteria you can match.
Let’s see this in action. Imagine you have two WAN connections, eth0 (ISP A) and eth1 (ISP B), and you want to send all traffic from your internal network (192.168.1.0/24) destined for 1.1.1.1 out through eth0, and everything else through eth1.
Here’s the nftables ruleset:
table ip mangle {
chain prerouting {
type filter hook prerouting priority mangle; policy accept;
ip saddr 192.168.1.0/24 ip daddr 1.1.1.1 meta mark set 0x1
}
}
table ip nat {
chain postrouting {
type nat hook postrouting priority 100; policy accept;
ip mark 0x1 ip daddr 0.0.0.0/0 oifname "eth0" masquerade
ip mark 0x0 ip daddr 0.0.0.0/0 oifname "eth1" masquerade
}
}
table ip route {
chain postrouting {
type route hook postrouting priority 0; policy accept;
ip mark 0x1 ip route change next-hop 10.0.0.1 dev eth0
ip mark 0x0 ip route change next-hop 10.0.1.1 dev eth1
}
}
In this setup:
- The
mangletable’spreroutingchain marks packets. If a packet originates from192.168.1.0/24and is going to1.1.1.1, it gets marked with0x1. - The
nattable’spostroutingchain performs Network Address Translation (NAT). Packets marked0x1are masqueraded outeth0, and unmarked packets are masqueraded outeth1. This ensures that the source IP of the outgoing packets is correctly translated for the respective ISP. - The
routetable’spostroutingchain handles the actual routing decision. If a packet has the mark0x1, it’s routed viaeth0(using10.0.0.1as the next hop). If it’s unmarked, it’s routed viaeth1(using10.0.1.1as the next hop).
This gives you fine-grained control. You could mark packets based on protocol, port, ToS bits, or even by matching against a specific set of destination IPs. The key is that the meta mark primitive allows you to attach a unique identifier to a packet that subsequent rules in different tables can inspect and act upon.
The nftables type route hook is critical here. Unlike filter or nat hooks which happen earlier, the route hook is specifically designed for modifying routing decisions after the kernel has already performed its initial routing lookup but before the packet is actually sent out. This allows you to override or refine the default routing behavior based on your packet marks.
The ip route change next-hop <gateway> dev <interface> command in the route table is the magic. It tells the kernel, "For packets matching this criteria (specifically, packets with this mark), don’t use the default route; instead, send them to <gateway> via <interface>." This is how you create policy-based routing.
The policy accept on all chains ensures that if no specific rule matches, the packet is not dropped by default within that chain’s processing.
Most people think of routing as solely a destination IP address lookup. The reality is that the kernel maintains multiple routing tables, and tools like ip rule and nftables allow you to select which table to use or even to modify the routing decision dynamically based on packet attributes, effectively creating policy-based routing.
The next step is often to introduce load balancing across multiple marked paths.