Nginx can act as a sophisticated load balancer for cloud services, but it’s often misunderstood as just a simple reverse proxy.
Here’s a look at Nginx in action, balancing traffic to a fleet of backend services running in a cloud environment.
Imagine you have a web application that consists of several microservices: users-service, products-service, and orders-service. These services are deployed on virtual machines or containers within your cloud provider (e.g., AWS EC2 instances, Kubernetes pods). You want to expose these services to the internet through a single entry point, handle SSL termination, and distribute traffic intelligently.
# nginx.conf
http {
# Define upstream groups for each service
upstream users-service {
server 10.0.1.10:8080;
server 10.0.1.11:8080;
# ... more user service instances
}
upstream products-service {
server 10.0.2.20:8081;
server 10.0.2.21:8081;
# ... more product service instances
}
upstream orders-service {
server 10.0.3.30:8082;
server 10.0.3.31:8082;
# ... more order service instances
}
# Define a server block for the main entry point
server {
listen 80;
server_name api.example.com;
# SSL configuration (optional, but recommended)
# listen 443 ssl;
# ssl_certificate /etc/nginx/ssl/api.example.com.crt;
# ssl_certificate_key /etc/nginx/ssl/api.example.com.key;
# Location block to route requests based on URI
location /users {
proxy_pass http://users-service;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}
location /products {
proxy_pass http://products-service;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}
location /orders {
proxy_pass http://orders-service;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}
# Default catch-all or health check endpoint
location / {
return 404 "Not Found";
}
}
}
In this setup, Nginx acts as the single public endpoint. When a request comes in for /users, Nginx forwards it to the users-service upstream group. It distributes the traffic across the available users-service instances using a round-robin algorithm by default. The proxy_set_header directives are crucial; they pass essential information like the original client’s IP address and the protocol (HTTP/HTTPS) to the backend services, which they might otherwise not see.
The problem this solves is multifaceted. Instead of each microservice needing its own public IP, SSL certificate, and load balancing logic, Nginx centralizes these concerns. It provides a unified API gateway, abstracts the complexity of the backend infrastructure, and allows for easier scaling and management of individual services. You can add or remove instances of users-service without changing the public API endpoint or affecting clients.
Internally, Nginx maintains a list of active servers for each upstream group. When a request arrives, it selects a server from the appropriate group based on the configured load balancing method. The proxy_pass directive tells Nginx where to send the request, and Nginx then establishes a connection to the chosen backend server, forwarding the request and streaming the response back to the client.
Beyond simple round-robin, Nginx supports other load balancing methods like least_conn (directs traffic to the server with the fewest active connections) and ip_hash (assigns a client to a specific server based on their IP address, useful for maintaining session affinity). You can also configure health checks to automatically remove unhealthy servers from the rotation.
The proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; directive is incredibly powerful because it constructs a list of IP addresses. If Nginx receives a request that already has an X-Forwarded-For header (meaning it passed through one or more proxies before reaching Nginx), Nginx appends the IP address of the immediate preceding proxy (or the original client if Nginx is the first proxy) to this list. This allows backend services to see the entire chain of proxies and the original client IP, which is vital for logging, analytics, and security.
When you use proxy_pass with a URI like proxy_pass http://users-service/some/path;, Nginx will strip the matched part of the location directive from the URI before forwarding. For example, if the request is /users/123 and the location is /users, Nginx will pass /123 to the backend. If you use proxy_pass http://users-service$request_uri;, the original URI /users/123 is passed through unchanged, which is often preferred when the backend services expect the full path.
The one thing most people don’t realize is how Nginx handles persistent connections. By default, Nginx tries to reuse existing connections to upstream servers for subsequent requests to the same server. This is configured via keepalive directives within the upstream block. For example, keepalive 32; tells Nginx to maintain up to 32 idle keepalive connections to each upstream server. This significantly reduces the overhead of establishing new TCP connections for every request, improving performance and reducing latency for your backend services.
The next concept to explore is integrating Nginx with cloud provider-managed load balancers, such as AWS Elastic Load Balancing (ELB) or Google Cloud Load Balancing.