Nginx is failing because the operating system is refusing to let it open any more file descriptors, a hard limit Nginx has hit.
This usually manifests as a cascade of errors in Nginx’s error log, often starting with [emerg] still could not bind() to ... or [error] accept() failed (24: Too many open files). The accept() error is the direct symptom; the bind() error is often a consequence of Nginx being unable to properly tear down existing connections because it can’t open files to manage them.
Here’s what’s really happening and how to fix it:
1. The Nginx worker_rlimit_nofile Setting
This is Nginx’s internal limit on the number of file descriptors each worker process can open. It’s often set too low by default.
-
Diagnosis: Check your
nginx.conf(or files included in it, typically inconf.d/orsites-available/). Look for aworker_rlimit_nofiledirective. If it’s not there, Nginx is likely using the OS default for processes, which is usually very low. -
Fix: Add or increase this directive in your
nginx.conffile, usually within themaincontext (outsidehttp,server,location). A common recommendation is65535.worker_rlimit_nofile 65535; http { # ... other http settings } -
Why it works: This tells Nginx to ask the operating system to allow its worker processes to open up to 65535 files. Nginx uses file descriptors for everything: client connections, upstream connections, log files, temporary files, etc.
2. System-Wide fs.file-max Limit
The operating system itself has a global limit on the total number of file descriptors it can have open across all processes. If this is too low, even a high worker_rlimit_nofile won’t help.
- Diagnosis: Run
sysctl fs.file-max. - Fix: Increase this value. You can do this temporarily with
sysctl -w fs.file-max=2097152or permanently by editing/etc/sysctl.conf(or a file in/etc/sysctl.d/) and adding the linefs.file-max = 2097152, then runningsysctl -p. - Why it works: This raises the ceiling for the entire system, allowing more total open files than the previous limit.
2097152is a common, generous value.
3. Per-User nofile Limit (ulimit)
Each user account (including the user Nginx runs as, often nginx or www-data) has its own limits on the number of open files. This is often configured via limits.conf.
-
Diagnosis:
- Find which user Nginx runs as:
ps aux | grep "nginx: worker process" - Check the limits for that user:
sudo su - <nginx_user> -s /bin/bash -c 'ulimit -n' - Inspect
/etc/security/limits.confand files in/etc/security/limits.d/for entries related to the Nginx user. Look fornofiledirectives.
- Find which user Nginx runs as:
-
Fix: Add or modify entries in
/etc/security/limits.conf(or a new file in/etc/security/limits.d/) to raise the hard and softnofilelimits for the Nginx user.# /etc/security/limits.d/99-nginx.conf <nginx_user> soft nofile 65535 <nginx_user> hard nofile 65535Replace
<nginx_user>with the actual username (e.g.,www-data). You’ll need to restart Nginx and potentially log out and back in as that user (or restart services that run under it) for these to take effect. -
Why it works: This directly increases the maximum number of files the Nginx user is allowed to open, overriding potentially lower system defaults or PAM configurations.
4. Systemd Service File Limits
If Nginx is managed by systemd, its service file can override ulimit settings.
-
Diagnosis:
- Find the Nginx systemd service file:
systemctl status nginxwill often show the path. It’s typically/lib/systemd/system/nginx.serviceor/etc/systemd/system/nginx.service. - Check the service file for
LimitNOFILEdirectives.
- Find the Nginx systemd service file:
-
Fix:
- Create an override file:
sudo systemctl edit nginx.service - Add the following to the override file:
[Service] LimitNOFILE=65535 LimitNOFILESoft=65535- Save and exit, then reload systemd:
sudo systemctl daemon-reload - Restart Nginx:
sudo systemctl restart nginx
- Create an override file:
-
Why it works: Systemd’s
LimitNOFILEdirective sets theulimitfor the process when it starts, effectively bypassing or reinforcing thelimits.confsettings.
5. Nginx Configuration: worker_connections
While not the root cause of hitting the OS limit, a too-low worker_connections can make it seem like you’re hitting a file descriptor limit prematurely. This setting determines how many simultaneous connections a single worker process can handle.
-
Diagnosis: Look for
worker_connectionsinside theeventsblock of yournginx.conf. -
Fix: Increase
worker_connections. A common value is4096or higher, depending on your expected traffic.events { worker_connections 4096; } -
Why it works: A higher
worker_connectionsallows each worker to handle more concurrent clients. Each client connection consumes a file descriptor. If this is too low, you might hit the OSnofilelimit sooner than you would with a higher setting, but the underlying OS limits are still the ultimate constraint.
6. Rogue Processes or Leaking Connections
Sometimes, the "too many open files" isn’t just Nginx itself, but a consequence of Nginx not closing things properly, or another process hogging descriptors.
- Diagnosis:
- Use
lsof -p $(pgrep nginx)to see all files open by Nginx processes. Sort by file type or count to identify patterns. - Use
lsof | awk '{ print $1 " " $2 " " $8 }' | sort | uniq -c | sort -nr | head -n 20to see the top processes with the most open files system-wide.
- Use
- Fix:
- If Nginx is leaking connections (e.g., due to upstream timeouts not being handled, or bugs in modules), ensure keepalive timeouts are reasonable and that Nginx is configured to properly close connections.
- If another process is the culprit, address that process’s resource usage.
- Why it works: Identifying and fixing the actual source of the open files, whether it’s Nginx’s behavior or another process, resolves the underlying issue.
After applying these fixes, remember to restart Nginx (sudo systemctl restart nginx or sudo service nginx restart) and potentially verify your ulimit settings by checking ulimit -n after logging in as the Nginx user.
The next error you might encounter, if you haven’t configured enough ephemeral ports, is [error] bind() to 0.0.0.0:80 failed (98: Address already in use).