A load balancer doesn’t just distribute traffic; it’s a sophisticated traffic cop that actively manipulates network flows based on a deep understanding of application state and client behavior.
Let’s look at a real-world scenario. Imagine a busy e-commerce site. Users are browsing, adding to carts, and checking out. The load balancer needs to ensure their experience is seamless, even as traffic spikes.
Here’s a simplified Nginx configuration for a load balancer:
http {
upstream backend_servers {
server 192.168.1.10:8080 weight=3 max_fails=3 fail_timeout=30s;
server 192.168.1.11:8080 weight=2 max_fails=3 fail_timeout=30s;
server 192.168.1.12:8080 backup;
}
server {
listen 80;
server_name example.com;
location / {
proxy_pass http://backend_servers;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection 'upgrade';
proxy_set_header Host $host;
proxy_cache_bypass $http_upgrade;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}
}
}
The upstream block defines your pool of backend servers. server 192.168.1.10:8080 weight=3 means this server will receive three times as much traffic as a server with weight=1. This is crucial for handling servers with different capacities. max_fails=3 sets the number of consecutive failed requests before a server is considered down, and fail_timeout=30s is the duration Nginx will wait before attempting to re-add a failed server to the pool. The backup directive marks 192.168.1.12:8080 as a standby server, only receiving traffic if all other servers in the upstream group are unavailable.
The server block listens on port 80 for incoming HTTP requests for example.com. The location / block handles all requests. proxy_pass http://backend_servers; is the core directive, forwarding requests to the defined upstream group. proxy_http_version 1.1; ensures HTTP/1.1 is used for backend connections, enabling features like keep-alive.
The proxy_set_header directives are vital for passing client information to the backend. Host $host; preserves the original Host header, important for virtual hosting on backends. X-Real-IP $remote_addr; adds the client’s IP address, and X-Forwarded-For $proxy_add_x_forwarded_for; appends it to any existing X-Forwarded-For headers, creating a chain of IPs if multiple proxies are involved. X-Forwarded-Proto $scheme; indicates whether the original client request was HTTP or HTTPS, which is essential if your load balancer handles SSL termination. proxy_cache_bypass $http_upgrade; is a specific optimization to prevent caching of WebSocket upgrade requests.
When a user requests /, Nginx consults the upstream configuration. It might send the request to 192.168.1.10 because it has a higher weight. If that server is slow or unresponsive, Nginx tracks its failures. After 3 failures within 30 seconds, it’s temporarily removed from rotation. If all primary servers fail, the backup server 192.168.1.12 is activated. The backend server receives the request, processes it, and sends the response back through Nginx, which then forwards it to the client, including the original Host header and the client’s IP address.
The sticky sessions mechanism, often configured with sticky cookie or ip_hash, ensures a client’s subsequent requests are always sent to the same backend server. This is critical for applications that maintain session state locally on the server. For example, sticky cookie srv_id expires=1h path=/; would create a cookie named srv_id that the load balancer uses to track which server handled the initial request, sending subsequent requests from that client to the same server for an hour.
Load balancers can also perform health checks. Nginx, for instance, can be configured to periodically ping backend servers using health_check directives (though this is more common in dedicated load balancer appliances or cloud provider services). In Nginx Plus, you’d see something like:
upstream backend_servers {
server 192.168.1.10:8080;
health_check uri=/healthz interval=5s fails=3 passes=2;
}
This tells Nginx Plus to send a GET request to /healthz on each backend server every 5 seconds. If 3 consecutive checks fail, the server is marked as unhealthy. It’s considered healthy again after 2 consecutive successful checks.
The interplay between weight, max_fails, fail_timeout, and health checks creates a robust system that can dynamically adapt to server availability and performance. Without these settings, a single unhealthy server could bring down the entire application, or traffic could be unevenly distributed, leading to performance bottlenecks on some servers while others sit idle.
Understanding how the load balancer rewrites or preserves headers like X-Forwarded-For is crucial for backend applications that need to log the actual client IP address for auditing or analytics. If these headers are not set or are incorrectly processed, the backend application will only see the load balancer’s IP, obscuring the original source of the traffic.
The next step in mastering load balancing is understanding advanced routing strategies, such as geo-based routing or content-aware routing, which allow for even more granular control over traffic distribution.