Articles

Locust Bottleneck Detection: Find Slow Endpoints Under Load (2026)

April 18, 2026 8 min read

When you’re running Locust tests and suddenly see a ton of 500 errors or your throughput plummets, it’s usually because one of your endpoints is drowning under load. Locust, by default, doesn’t explicitly tell you which endpoint is the culprit, but it’s the most common reason your entire test grinds to a halt.

Let’s say your API has a /users endpoint and a /products endpoint. You’re hitting them with 1000 concurrent users, and suddenly your average response time for /users spikes to 5 seconds while /products stays at 200ms. The system isn’t failing because Locust itself is broken; it’s failing because your application can’t keep up with the requests hitting that specific /users endpoint, and the server is starting to reject them or respond incredibly slowly.

Here’s how to find those slowpokes:

1. The "All Endpoints Are Slow" Illusion

Problem: Your Locust Web UI shows a high average response time, but drilling down into individual endpoint statistics seems to indicate they’re all performing okay-ish. This is misleading. A few very slow requests can skew the average so badly that the individual statistics look deceptive.

Diagnosis: Use the locust --headless --host=http://localhost:8080 --run-time 1m --only-summary --print-stats command. This will dump raw statistics to your terminal at the end of the run. Look for the Total and Failures sections. If Total response times are high (e.g., > 1s) and Failures are significant, you have a problem. The individual endpoint stats might still look okay if the load is distributed, but the overall picture is grim.

Cause: Often, this points to a resource contention issue on your server that isn’t tied to a single endpoint but affects many, or a fundamental network bottleneck upstream of your application.

Fix: This is where you start looking outside Locust.

  • Check Server CPU/Memory: On your application server, run top or htop. If CPU is consistently above 80% or memory is exhausted, scale up your server or optimize your application’s resource usage.
  • Database Connection Pool: If your application uses a database, check its connection pool size. If it’s exhausted, requests will queue up. On PostgreSQL, check max_connections in postgresql.conf and pg_stat_activity. Increase it if necessary (e.g., from 100 to 200) and restart the DB.
  • Load Balancer/Reverse Proxy: If you have an Nginx or HAProxy in front, check its logs and resource usage. A misconfigured rate limit or insufficient worker processes can choke traffic. For Nginx, look at worker_processes in nginx.conf and ensure it matches your CPU cores; check worker_connections too.

Why it works: These issues prevent your application servers from processing requests efficiently, leading to cascading slowdowns and failures across the board.

2. The "One Endpoint is a Black Hole" Scenario

Problem: The Locust Web UI clearly shows one specific endpoint (e.g., /api/v1/process_data) has a vastly higher average response time and/or failure rate than others.

Diagnosis: In the Locust Web UI, under the "Statistics" tab, sort by "Response Times (Average)" descending. The endpoint at the top is your prime suspect. Note its average response time and failure count.

Cause: This is the most common scenario. A specific piece of code, a database query, or an external service call within that endpoint is the bottleneck.

Fix:

  • Application Profiling: Use your application’s built-in profiling tools. For Python/Django/Flask, cProfile or py-spy can pinpoint slow functions. For Java, use JProfiler or VisualVM. For Node.js, node --prof. Identify the function or method taking the longest.
    • Example (Python): If profiling shows a slow_database_query() function is the culprit, investigate that query. EXPLAIN ANALYZE on the SQL query in your database will reveal if it’s missing an index, has a bad execution plan, or is performing a full table scan.
    • Example (Fix): Add an index to the users table on the email column if queries against WHERE email = '...' are slow: CREATE INDEX idx_users_email ON users (email);. This allows the database to find rows much faster.
  • External Service Latency: If the endpoint calls an external API (e.g., a payment gateway, a third-party data provider), that service might be slow. Add logging before and after the external call within your endpoint to measure its specific latency.
    • Example (Fix): If an external call consistently takes 2 seconds, consider making it asynchronous (e.g., using Celery in Python, or a message queue) so your main endpoint can respond quickly while the background task completes.
  • Inefficient Data Structures/Algorithms: The code itself might be doing too much work. A common culprit is iterating over large lists multiple times or using O(n^2) algorithms where O(n log n) or O(n) would suffice.
    • Example (Fix): If you’re searching through a list of 10,000 items repeatedly, convert it to a set or dict (hash table) first for O(1) average-case lookups.

Why it works: By identifying and optimizing the specific code path, database interaction, or external dependency causing the slowdown, you remove the direct bottleneck.

3. The "Thundering Herd" Problem

Problem: Your application works fine under moderate load, but as soon as you ramp up users, a specific endpoint (or the entire system) becomes exponentially slower or starts failing.

Diagnosis: Observe the response times and failure rates in Locust as you gradually increase the user count. If you see a sharp, non-linear increase in latency or failures past a certain user threshold (e.g., 500 users), you’re likely hitting a resource limit that only manifests under high concurrency.

Cause: This often points to issues with:

  • Lock Contention: Multiple threads or processes trying to acquire the same lock, causing threads to block indefinitely.
  • Thread Pool Exhaustion: Your application server (e.g., Tomcat, Gunicorn) has a finite number of worker threads. Once they’re all busy, new requests have to wait for a thread to become free.
  • External Dependency Limits: A downstream service or database connection pool has a hard limit that gets hit simultaneously by many concurrent requests.

Fix:

  • Optimize Locking: Review your code for critical sections protected by locks. Can the lock be held for a shorter duration? Can the critical section be made lock-free using atomic operations or concurrent data structures?
  • Increase Worker Threads/Processes: Tune your application server’s configuration. For Gunicorn (Python), increase workers (e.g., gunicorn -w 8 myapp:app). For Tomcat, adjust maxThreads in server.xml. This needs to be balanced with available server CPU/memory.
  • Asynchronous Processing: If requests involve waiting (I/O bound), use asynchronous programming models (async/await in Python/JS, Goroutines in Go) or offload work to background queues (Celery, RabbitMQ, Kafka). This frees up your main request-handling threads.

Why it works: By either reducing contention, providing more concurrent processing units, or making operations non-blocking, you prevent requests from piling up and timing out.

4. The "Network is the Bottleneck" Scenario

Problem: Locust and your application servers seem fine, but requests are timing out, and network monitoring tools show high latency or packet loss between Locust clients and the target host, or between application components.

Diagnosis:

  • Locust Client Network: Use ping and traceroute from the machine running your Locust clients to the target host. High latency or packet loss here points to a network issue between your test environment and the application.
  • Application Server Network: Use ping and traceroute from your application servers to any databases or external services they depend on.
  • Server Network Interface: Check network interface statistics on your application servers for errors, dropped packets, or saturation using netstat -s or tools like iftop.

Cause: Under load, network infrastructure (routers, switches, firewalls, NICs) can become saturated or misconfigured.

Fix:

  • Bandwidth: If your network link is saturated, you may need to provision more bandwidth.
  • Firewall/Security Groups: Ensure that firewalls or security group rules aren’t rate-limiting or blocking traffic under load. Check logs on these devices.
  • NIC Offloading: Sometimes, NIC offloading features can cause issues under extreme load. Temporarily disabling them (ethtool -K <interface> tx off rx off) can help diagnose.
  • MTU Mismatches: Inconsistent Maximum Transmission Unit (MTU) sizes across network devices can cause fragmentation and performance degradation.

Why it works: Ensures that data packets can travel efficiently and reliably between all components involved in the request lifecycle.

5. The "Database is Drowning" Scenario

Problem: Your application seems okay, but specific database queries (often triggered by a particular endpoint) are taking ages.

Diagnosis:

  • Application Logs: Check your application logs for slow query warnings. Many frameworks log queries exceeding a certain threshold (e.g., 500ms).
  • Database Performance Monitoring: Use your database’s built-in tools. For PostgreSQL, pg_stat_statements extension is invaluable for identifying the slowest queries. For MySQL, SHOW PROCESSLIST and EXPLAIN are key.
  • Locust Statistics: As mentioned before, the Locust statistics tab will highlight endpoints that are slow, and you can then correlate those to the database operations they perform.

Cause: Missing indexes, inefficient queries, database server resource exhaustion (CPU, RAM, I/O), or lock contention within the database.

Fix:

  • Add Indexes: Use EXPLAIN ANALYZE on slow queries to identify missing indexes. CREATE INDEX index_name ON table_name (column_name);
  • Optimize Queries: Rewrite queries to be more efficient. Avoid SELECT *, use JOINs appropriately, and minimize subqueries where possible.
  • Database Tuning: Increase buffer sizes, adjust query planner parameters, and ensure sufficient hardware resources for the database server.
  • Connection Pooling: Ensure your application is using a connection pool and that its size is adequate but not excessive.

Why it works: Directly addresses the performance of the database, which is often the backend for many application operations.

The next error you’ll hit after fixing these is likely a ResourceWarning or an OutOfMemoryError if you’ve scaled up too aggressively without addressing the root cause, or perhaps a ConnectionRefusedError if a downstream service you inadvertently created a dependency on during your fixes is now failing.

Want structured learning?

Take the full Locust course →