The RST_STREAM error means a client or server abruptly terminated a specific HTTP/2 stream, usually due to an unrecoverable issue on that stream’s path.
Common Causes and Fixes for RST_STREAM
-
Large or Malformed Frames: An endpoint received a frame that’s too large for its configured limits, or the frame data itself is corrupted. This is the most frequent culprit.
- Diagnosis: Check server logs (e.g., Nginx, Apache) for messages like "client intended to send too large frame body" or "invalid frame size." On the client side, browser developer tools might show
ERR_SPDY_PROTOCOL_ERROR. - Fix (Nginx): Increase
client_max_body_sizein yournginx.confor site-specific configuration. For example,client_max_body_size 100m;. This tells Nginx to accept larger request bodies. - Fix (Apache): Adjust
LimitRequestBodyin yourhttpd.confor virtual host configuration. For example,LimitRequestBody 104857600(100MB). - Why it works: These directives set the maximum allowed size for request bodies, preventing the server from crashing or erroring out when it receives a large, valid request.
- Fix (Client-side): If you control the client, ensure it’s not sending excessively large frames or that it’s correctly fragmenting large data.
- Diagnosis: Check server logs (e.g., Nginx, Apache) for messages like "client intended to send too large frame body" or "invalid frame size." On the client side, browser developer tools might show
-
Stream Dependency Issues: An invalid stream dependency or weight was sent, confusing the connection’s flow control.
- Diagnosis: Look for logs mentioning "invalid stream dependency" or "stream state error."
- Fix: This is rarely a server configuration issue; it’s more likely a bug in the client or intermediate proxy software. Ensure all HTTP/2 client libraries and proxies are up-to-date.
-
Connection Keep-Alive Timeout: The client or server closed the underlying TCP connection due to inactivity, and a new stream was attempted on a stale connection.
- Diagnosis: Check server logs for connection closure messages. Client-side, you might see a brief connection drop before the error.
- Fix (Nginx): Increase
keepalive_timeoutinnginx.conf. Example:keepalive_timeout 75s;. - Fix (Apache): Adjust
KeepAliveTimeoutinhttpd.conf. Example:KeepAliveTimeout 75. - Why it works: This extends the period the server will keep an idle TCP connection open, allowing more time for subsequent requests before a new connection must be established.
-
Resource Exhaustion (Server-side): The server ran out of memory, file descriptors, or other resources while processing a stream.
- Diagnosis: Monitor server system metrics (CPU, RAM, open file descriptors) during periods of high load. Check system logs (
/var/log/syslogorjournalctl) for OOM killer messages or "too many open files" errors. - Fix: Increase server resources (RAM, CPU). Adjust OS limits for open file descriptors (
ulimit -n). For example, setulimit -n 65536in/etc/security/limits.conf. - Why it works: By providing more resources and allowing more file handles, the server can cope with a higher number of concurrent connections and streams without failing.
- Diagnosis: Monitor server system metrics (CPU, RAM, open file descriptors) during periods of high load. Check system logs (
-
Application Logic Error: The application code itself encountered an unrecoverable error while processing a request associated with a specific stream.
- Diagnosis: Examine application logs for exceptions or unhandled errors occurring precisely when the
RST_STREAMhappens. - Fix: Debug and fix the application code. This might involve error handling, input validation, or resource management within the application.
- Diagnosis: Examine application logs for exceptions or unhandled errors occurring precisely when the
-
H2C (HTTP/2 Cleartext) Issues: If using HTTP/2 without TLS (H2C), proxies or clients might not correctly negotiate or handle the upgrade.
Common Causes and Fixes for PROTOCOL_ERROR
PROTOCOL_ERROR is a more general HTTP/2 error indicating a violation of the protocol’s rules, often due to malformed frames, incorrect header compression, or invalid settings.
-
Invalid Header Compression (HPACK): The HPACK decompressor on one end failed to reconstruct the headers sent by the other. This can happen with malformed HPACK instructions.
- Diagnosis: This is notoriously hard to debug directly. Look for
RST_STREAMwith a specific error code or general connection instability. Browser developer tools might showERR_SPDY_PROTOCOL_ERROR. Server logs might have generic "HPACK decoding error." - Fix: Ensure all HTTP/2 implementations (server, client, libraries, proxies) are up-to-date and conform to the latest RFCs. Sometimes, a specific library version can cause this.
- Diagnosis: This is notoriously hard to debug directly. Look for
-
Incorrect
SETTINGSFrame: An endpoint sent aSETTINGSframe with invalid parameters or an invalid sequence.- Diagnosis: Check logs for "invalid SETTINGS frame" or similar.
- Fix: This is usually a bug in an HTTP/2 library or proxy. Ensure all components are updated. If you’re implementing an HTTP/2 server or client, meticulously review your
SETTINGSframe generation and handling logic.
-
Flow Control Violations: An endpoint sent data exceeding the advertised window size for a stream or the connection.
- Diagnosis: Server logs might indicate "flow control window exceeded."
- Fix (Server-side): Ensure your HTTP/2 server’s flow control logic is correctly implemented. For Nginx, this is generally handled internally, but custom modules or extreme load could trigger issues. For custom servers, review your
WINDOW_UPDATEframe generation.
-
Frame Type Mismatch: An endpoint sent a frame type that is not expected in the current state of the stream or connection.
- Diagnosis: Logs might show "received unexpected frame type."
- Fix: Again, this points to a bug in the HTTP/2 stack of one of the communicating parties. Update libraries and software.
Common Causes and Fixes for GOAWAY
GOAWAY is sent by an endpoint to indicate that it’s shutting down the connection or has encountered an error that prevents it from continuing to process streams on that connection. It signals the last stream ID it was able to process.
-
Server Shutdown/Restart: The most common reason is a planned or unplanned server restart. The server sends
GOAWAYto gracefully close existing connections.- Diagnosis: Check server process status and logs for restart events. If you’re managing the server, this is usually self-explanatory.
- Fix: If planned, ensure clients are aware and can reconnect. If unplanned, investigate the cause of the crash/shutdown.
-
Protocol Violation (Endpoint’s Perspective): The server detected a protocol violation from the client and is terminating the connection.
- Diagnosis: The
GOAWAYframe will often include aPROTOCOL_ERRORorREFUSED_STREAMerror code. Check server logs for the specific reason the client’s behavior was deemed a violation. - Fix: Debug the client application or proxy to ensure it’s adhering to HTTP/2 protocol standards.
- Diagnosis: The
-
Resource Limits Reached: The server is experiencing resource exhaustion (memory, CPU, connections) and is proactively closing connections to prevent a full crash.
- Diagnosis: Monitor server resource utilization. Look for system logs indicating high load, OOM killer activity, or connection limits being hit.
- Fix: Scale server resources (CPU, RAM) or optimize application performance to reduce resource consumption. Adjust connection limits if applicable (e.g.,
worker_connectionsin Nginx).
-
Idle Connection Timeout: The HTTP/2 connection has been idle for too long, and the server is closing it according to its configuration.
- Diagnosis: Check server
keepalive_timeoutor idle connection settings. - Fix: Increase the
keepalive_timeouton the server. For Nginx,keepalive_timeout 75s;. For Apache,KeepAliveTimeout 75.
- Diagnosis: Check server
-
H2 to H1 Downgrade or Proxy Issues: An intermediary proxy might be terminating the HTTP/2 connection and downgrading it to HTTP/1.1, or it might be malfunctioning.
- Diagnosis: If
GOAWAYappears only when traversing a specific proxy, investigate the proxy’s logs and configuration. - Fix: Ensure the proxy is correctly configured for HTTP/2 and that its software is up-to-date.
- Diagnosis: If
The next error you’ll likely encounter after resolving these is a REFUSED_STREAM if a client attempts to create a new stream on a connection that has already been shut down by a GOAWAY frame.