Nmap’s FIN, NULL, and Xmas scans can sneak past firewalls by exploiting how TCP stacks are supposed to handle malformed packets.
Let’s see them in action. Imagine we have a target server, 192.168.1.100, and we want to see what’s open on port 80.
# Example of a FIN scan
nmap -sF 192.168.1.100 -p 80
# Example of a NULL scan
nmap -sN 192.168.1.100 -p 80
# Example of an Xmas scan
nmap -sX 192.168.1.100 -p 80
The magic here isn’t about sending a packet that gets through; it’s about observing the response (or lack thereof) to a packet that shouldn’t be answered.
The Problem: State-Inspecting Firewalls
Most modern firewalls are stateful. They track established connections. When you send a SYN packet to open a port, the firewall sees it, expects a SYN-ACK, and lets it pass. If it sees a SYN-ACK without a prior SYN, it drops it. This is great for blocking unsolicited traffic.
However, these scans exploit a quirk in the TCP RFC (specifically RFC 793). For ports that are closed, a TCP stack should respond to a FIN, NULL, or Xmas packet with a RST (reset) packet. For ports that are open, the RFC states that these malformed packets should be silently ignored.
How FIN, NULL, and Xmas Scans Work
-
FIN Scan (
-sF): Sends a TCP packet with only the FIN flag set.- Target is CLOSED: The target’s TCP stack should respond with a RST, ACK.
- Target is OPEN: The target’s TCP stack should ignore the packet.
- Firewall Observation: If Nmap receives no response (or an ICMP unreachable message, which is less common for this specific scan), it infers the port is OPEN. If it receives a RST, ACK, it infers the port is CLOSED.
-
NULL Scan (
-sN): Sends a TCP packet with no flags set.- Target is CLOSED: The target’s TCP stack should respond with a RST, ACK.
- Target is OPEN: The target’s TCP stack should ignore the packet.
- Firewall Observation: Same as FIN scan: no response implies OPEN, RST implies CLOSED.
-
Xmas Scan (
-sX): Sends a TCP packet with the FIN, PSH, and URG flags set (named "Xmas" because it lights up like a Christmas tree).- Target is CLOSED: The target’s TCP stack should respond with a RST, ACK.
- Target is OPEN: The target’s TCP stack should ignore the packet.
- Firewall Observation: Again, no response implies OPEN, RST implies CLOSED.
These scans are stealthy because they don’t complete the three-way handshake (SYN, SYN-ACK, ACK). A stateful firewall typically only tracks packets involved in a handshake or already established connections. Since these scans send packets that look like they’re part of an existing, but not yet established, connection, or are simply malformed, they might not trigger the firewall’s logging or blocking rules that are designed for SYN scans.
The Catch: Not All Systems Obey the RFC
The behavior described above is the RFC-compliant way. However, many modern operating systems, especially Windows, don’t strictly adhere to this. They might send a RST to both open and closed ports when receiving a FIN, NULL, or Xmas packet.
- Windows Behavior: Often, Windows will send a RST to an open port for these scans. This means Nmap will incorrectly report the port as CLOSED.
- Linux/Unix Behavior: Generally more compliant, but variations exist.
This is why Nmap often reports "open|filtered" for these scan types. It means Nmap didn’t get a definitive RST, so it thinks the port might be open, but it’s also possible a firewall is dropping the packets, or the host is simply ignoring them.
Practical Usage and Limitations
- When to Use: These scans are best used against systems you suspect might be filtering standard SYN scans but are less likely to have custom firewall rules to block these specific malformed packet types. They are also good for identifying hosts that are non-responsive to these types of probes.
- Firewall Evasion: The primary goal is to bypass firewalls that only track SYN packets. If a firewall is configured to drop any unsolicited TCP packet to a closed port, or to block packets with unusual flag combinations, these scans will be detected.
- Interpreting Results:
open|filtered: This is the most common and ambiguous result. It means Nmap didn’t receive a RST, so the port is either open or a firewall is blocking the probe.closed: Nmap received a RST, ACK, indicating the port is definitely closed.filtered: Nmap received an ICMP error (like "port unreachable") indicating the port is filtered.
To get a definitive answer on whether a port is open, you’ll often need to fall back to a full SYN scan (-sS) or TCP Connect scan (-sT), which are more reliable but also more easily detected.
The next hurdle you’ll face is understanding how ICMP error messages are handled by network devices, and how Nmap uses them to infer port states.