The rp_filter setting is causing your system to drop legitimate incoming packets because it’s incorrectly identifying them as spoofed.
Common Causes and Fixes for rp_filter Dropping Packets
The Reverse Path Forwarding (RPF) check, enforced by rp_filter, is a security mechanism designed to prevent IP address spoofing. When a packet arrives on an interface, rp_filter checks if the source IP address of that packet is reachable via the same interface on which it arrived. If it’s not, the packet is dropped. This is usually a good thing, but it can cause problems in certain network configurations.
Here are the most common reasons rp_filter might be dropping your packets and how to fix them:
-
Asymmetric Routing:
- Diagnosis: This is the most frequent culprit. Asymmetric routing occurs when traffic from host A to host B takes a different path than traffic from host B back to host A. If your server receives traffic from a client on interface
eth0, but the route back to that client’s IP address points out of interfaceeth1,rp_filterwill see the incoming packet oneth0and check the routing table. It will find that the return path to the client’s IP is viaeth1. Since the packet arrived oneth0and the return path iseth1,rp_filterconsiders this an invalid, potentially spoofed, source and drops the packet. - Fix: The simplest fix is to disable
rp_filterfor the affected interface.
This tells the kernel to stop performing the RPF check on incoming packets arriving onsudo sysctl -w net.ipv4.conf.eth0.rp_filter=0eth0. - Why it works: By setting
rp_filterto0(disabled), you are explicitly telling the kernel to trust that incoming packets oneth0are legitimate, even if the return route isn’t through the same interface. This bypasses the RPF check that was causing the drops.
- Diagnosis: This is the most frequent culprit. Asymmetric routing occurs when traffic from host A to host B takes a different path than traffic from host B back to host A. If your server receives traffic from a client on interface
-
Multiple Default Routes:
- Diagnosis: If your system has more than one default route (
0.0.0.0/0), the kernel might choose an unexpected interface as the "best" route for a particular source IP. When a packet arrives on one interface, and the kernel’s selected default route for that source IP points out a different interface,rp_filterwill drop the packet. This is a variation of asymmetric routing, but specifically involving default gateways. - Fix: Ensure you have only one active default route. If you need multiple gateways for redundancy, use advanced routing policies or specific interface-based routing, but avoid having multiple unqualified default routes.
# Check current routes ip route show # Example: Remove a problematic default route (replace 192.168.1.254 with your gateway IP) sudo ip route del default via 192.168.1.254 dev eth0 - Why it works: Eliminating the ambiguity of multiple default routes ensures that the kernel has a single, clear path to use for outbound traffic, which in turn makes the RPF check more predictable and less likely to drop valid packets.
- Diagnosis: If your system has more than one default route (
-
Source-Specific Routing (Advanced):
- Diagnosis: In more complex setups, you might intentionally configure source-specific routing where traffic originating from a particular source IP should egress a specific interface, even if it arrives on another. For example, traffic from
10.10.10.0/24should go outeth1, but it might arrive oneth0. Ifrp_filteris enabled (even in strict mode), it will see the packet oneth0and check if the source10.10.10.0/24is reachable viaeth0. If your source-specific route directs it toeth1, the RPF check will fail. - Fix: You can enable
rp_filterinloosemode for the interface.
Note: This is the default forsudo sysctl -w net.ipv4.conf.eth0.rp_filter=1rp_filter=1. If it was previously set to2(strict), changing it to1(loose) is the fix. - Why it works:
rp_filter=1(loose mode) checks if the source IP is reachable via any interface, not just the ingress interface. This allows source-specific routing to function correctly because the packet’s source IP might be reachable viaeth1(due to your source-specific route), even though the packet arrived oneth0.
- Diagnosis: In more complex setups, you might intentionally configure source-specific routing where traffic originating from a particular source IP should egress a specific interface, even if it arrives on another. For example, traffic from
-
Virtualization/Container Environments:
- Diagnosis: In environments like Docker, Kubernetes, or KVM, network traffic often traverses multiple layers of virtual interfaces and routing. The host kernel’s
rp_filtermight not understand the complex routing within the virtual network, leading it to incorrectly drop packets destined for containers or VMs. For instance, a packet might arrive on the host’s physical interface, be routed by the host to a bridge, then to a veth pair for a container. If the host’srp_filterfor the physical interface doesn’t see a direct route back to the source IP through that same physical interface, it can drop the packet. - Fix: Similar to asymmetric routing, disabling
rp_filteron the host’s relevant network interfaces is often necessary.# Assuming 'br0' is the bridge interface for your VMs/containers sudo sysctl -w net.ipv4.conf.br0.rp_filter=0 # Also potentially for the physical interface sudo sysctl -w net.ipv4.conf.eth0.rp_filter=0 - Why it works: The complexity of virtual networking makes it difficult for the host’s RPF check to accurately determine the correct return path. Disabling
rp_filteron the host interfaces that connect to the virtual network allows traffic to flow without being prematurely dropped by the host’s security checks.
- Diagnosis: In environments like Docker, Kubernetes, or KVM, network traffic often traverses multiple layers of virtual interfaces and routing. The host kernel’s
-
Network Address Translation (NAT) on the Gateway:
- Diagnosis: If your router or firewall performs NAT on traffic originating from your server, the source IP address seen by the destination might be different from the IP address your server uses internally. When your server sends a packet, it uses its internal IP. The router NATs it to an external IP. If
rp_filteris enabled on the server and the return traffic comes back to the external IP, the server might check its routing table and find that the route back to the external IP is not the same interface it originally sent the packet out on (because the packet originally used an internal IP). This is a less common scenario but can occur. - Fix: This is best addressed by ensuring your NAT device correctly handles return traffic and that your server’s routing is consistent. If the issue persists, disabling
rp_filteron the server’s outgoing interface might be required.sudo sysctl -w net.ipv4.conf.eth0.rp_filter=0 - Why it works: If the RPF check on the server is causing issues due to NAT complexities, disabling it removes that layer of scrutiny, allowing the NATed traffic to pass through.
- Diagnosis: If your router or firewall performs NAT on traffic originating from your server, the source IP address seen by the destination might be different from the IP address your server uses internally. When your server sends a packet, it uses its internal IP. The router NATs it to an external IP. If
-
Misconfigured Static Routes:
- Diagnosis: Incorrectly configured static routes can lead to situations where the kernel believes a source IP is reachable via an interface that it’s not, or vice-versa. This can confuse
rp_filter. For example, a static route might point a large subnet to the wrong interface. - Fix: Carefully review all static routes using
ip route show. Ensure they accurately reflect your network topology and that there are no overlapping or contradictory routes, especially those involving the interface where packets are being dropped.# Example: Remove a misconfigured static route (replace with actual route details) sudo ip route del 192.168.5.0/24 via 10.0.0.254 dev eth1 - Why it works: Correcting static routes ensures that the kernel’s routing table accurately represents network reachability, allowing
rp_filterto make correct RPF decisions.
- Diagnosis: Incorrectly configured static routes can lead to situations where the kernel believes a source IP is reachable via an interface that it’s not, or vice-versa. This can confuse
After applying these fixes, you might encounter issues with ARP resolution if your network is particularly chatty or has old devices.