The most surprising truth about network flow control is that it’s not about preventing congestion; it’s about managing it gracefully so that the network doesn’t collapse.
Imagine two programs, sender.py and receiver.py, on separate machines. sender.py wants to send a lot of data to receiver.py.
# sender.py
import socket
import time
HOST = 'receiver_ip_address' # Replace with actual receiver IP
PORT = 12345
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
s.connect((HOST, PORT))
print(f"Connected to {HOST}:{PORT}")
data = b'A' * 1024 * 1024 # 1MB of data
try:
while True:
bytes_sent = s.send(data)
if bytes_sent == 0:
print("Connection closed by receiver.")
break
print(f"Sent {bytes_sent} bytes.")
# time.sleep(0.01) # Uncommenting this would simulate slower sending
except BrokenPipeError:
print("Connection broken (receiver likely closed).")
except Exception as e:
print(f"An error occurred: {e}")
print("Sender finished.")
# receiver.py
import socket
import time
HOST = '0.0.0.0' # Listen on all available interfaces
PORT = 12345
BUFFER_SIZE = 1024 # Small buffer to demonstrate flow control
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
s.bind((HOST, PORT))
s.listen()
print(f"Listening on {HOST}:{PORT}")
conn, addr = s.accept()
with conn:
print(f"Connected by {addr}")
total_received = 0
while True:
data = conn.recv(BUFFER_SIZE)
if not data:
print("Connection closed by sender.")
break
total_received += len(data)
# Simulate slow processing by the receiver
# time.sleep(0.001) # Uncommenting this would make receiver slower
# print(f"Received {len(data)} bytes. Total: {total_received}")
If sender.py is very fast and receiver.py is slow (or the network path between them is narrow), the sender will try to shove data faster than the receiver can consume it or the network can carry it. This is where flow control kicks in.
At its core, flow control is a mechanism for the receiver to tell the sender to slow down. The most common form in TCP/IP is window-based flow control. Each TCP connection has a "receive window" size. This window represents the amount of unacknowledged data the receiver is willing to accept.
When the sender sends data, it enters the receiver’s buffer. If the receiver’s buffer starts to fill up, it advertises a smaller receive window to the sender. If the sender has sent more data than the current window size allows, it will stop sending until it receives an acknowledgment (ACK) from the receiver that contains an updated, larger window size. This effectively creates a feedback loop, preventing the sender from overwhelming the receiver’s buffer and, by extension, the network path.
The socket module in Python, when using TCP (socket.SOCK_STREAM), handles this TCP-level window-based flow control automatically. You don’t typically write explicit flow control code in your application. However, you can observe its effects and influence them indirectly.
The receiver.py script above, by using a small BUFFER_SIZE for conn.recv(BUFFER_SIZE), indirectly affects how quickly the receiver processes data. If the receiver’s application logic (the code after conn.recv) is slow, the TCP receive buffer will fill up. The operating system on the receiver’s machine will then reduce the advertised TCP window size to the sender.
Consider the sender.py script. The s.send(data) call will block if the TCP send buffer is full. This buffer fills up when the application tries to send data faster than the network can transmit it and faster than the receiver acknowledges it. The send() call only returns once the data has been successfully handed off to the operating system’s network stack. If the TCP window is full, send() will wait.
You can observe this behavior using netstat on Linux/macOS or Get-NetTCPConnection in PowerShell on Windows.
On Linux/macOS, while sender.py is running and the receiver is slow:
netstat -anp tcp | grep sender_pid
Look for the Recv-Q (receive queue) and Send-Q (send queue) for the established connection. If the receiver is slow, Send-Q will grow. The TCP window size is dynamically adjusted by the OS, but the application-level buffer is what the send() call interacts with.
The actual TCP window size is negotiated during the TCP handshake and can be up to 65535 bytes for standard TCP. However, modern operating systems use TCP Window Scaling (defined in RFC 7323) to allow much larger windows, often in the megabytes, to accommodate high-bandwidth, high-latency networks. This scaling factor is negotiated during the handshake. You can see it in netstat -s output under TcpExt: ... TCP: ... sections related to scaling.
If you want to experimentally see the effect of a small receive buffer on the receiver side:
In receiver.py, change BUFFER_SIZE = 1024 to BUFFER_SIZE = 4096. This gives the receiver a slightly larger chunk to process each time. If the sender is very fast, you might see it send slightly more data before the TCP window forces it to slow down.
The most impactful lever an application developer has over TCP flow control is not directly manipulating the TCP window, but rather ensuring the application consumes data as fast as it’s received. If receiver.py had a time.sleep(0.1) after conn.recv(), the Send-Q on the sender would quickly fill up, and s.send() would block, demonstrating the flow control mechanism in action. Conversely, if the sender application were to call s.send() in a tight loop without any delays, and the receiver application were also very fast, the Send-Q would remain low, and s.send() would rarely block.
The maximum segment size (MSS) is another related concept. It’s the largest amount of data, specified in bytes, that TCP is willing to accept in a single segment. This is advertised by the receiver during the handshake and is crucial for efficient data transfer, as it dictates how much data can fit into a single IP packet. Path MTU discovery helps determine the optimal MSS to avoid IP fragmentation.
Ultimately, flow control is about preventing packet loss due to buffer overflows. When a buffer overflows, packets are dropped. Losing packets triggers retransmissions, which further exacerbates congestion. By slowing down the sender before buffers overflow, flow control keeps the network stable.
The next logical step after understanding flow control is exploring congestion control, which is TCP’s sophisticated mechanism for dynamically adjusting sending rates based on observed network conditions like packet loss and latency.