Articles

JMeter Baseline Tests: Establish Performance Benchmarks (2026)

April 18, 2026 5 min read

The most surprising thing about JMeter baseline tests is that they aren’t about finding the fastest possible performance; they’re about defining the normal performance, so you can spot when things go wrong.

Let’s see JMeter in action. Imagine we’re testing a simple API endpoint that returns user details.

Here’s a basic JMeter test plan.

<?xml version="1.0" encoding="UTF-8"?>
<jmeterTestPlan version="1.2" properties="5.0" jmeter="5.5">
  <hashTree>
    <TestPlan guiclass="TestPlanGui" testclass="TestPlan" testname="Baseline User API Test" enabled="true">
      <stringProp name="TestPlan.comments"></stringProp>
      <boolProp name="TestPlan.functional_mode">false</boolProp>
      <boolProp name="TestPlan.serialize_threadgroups">false</boolProp>
      <elementProp name="TestPlan.user_defined_variables" elementType="Arguments" guiclass="ArgumentsPanel" testclass="Arguments" enabled="true">
        <collectionProp name="Arguments.arguments"/>
      </elementProp>
      <stringProp name="TestPlan.user_define_classpath"></stringProp>
    </TestPlan>
    <hashTree>
      <ThreadGroup guiclass="ThreadGroupGui" testclass="ThreadGroup" testname="Users Thread Group" enabled="true">
        <stringProp name="ThreadGroup.on_sample_error">continue</stringProp>
        <elementProp name="ThreadGroup.main_controller" elementType="LoopController" guiclass="LoopControlPanel" testclass="LoopController" enabled="true">
          <boolProp name="LoopController.continue_forever">false</boolProp>
          <stringProp name="LoopController.loops">1</stringProp>
        </elementProp>
        <stringProp name="ThreadGroup.num_threads">10</stringProp>
        <stringProp name="ThreadGroup.ramp_time">1</stringProp>
        <boolProp name="ThreadGroup.scheduler">false</boolProp>
        <stringProp name="ThreadGroup.duration"></stringProp>
        <stringProp name="ThreadGroup.delay"></stringProp>
      </ThreadGroup>
      <hashTree>
        <HTTPSamplerProxy guiclass="HttpSamplerProxyGui" testclass="HTTPSamplerProxy" testname="GET /users/{id}" enabled="true">
          <elementProp name="HTTPsampler.Arguments" elementType="Arguments" guiclass="HTTPArgumentsPanel" testclass="Arguments" enabled="true">
            <collectionProp name="Arguments.arguments"/>
          </elementProp>
          <stringProp name="HTTPSamplerProxy.domain">localhost</stringProp>
          <stringProp name="HTTPSamplerProxy.port">8080</stringProp>
          <stringProp name="HTTPSamplerProxy.protocol">http</stringProp>
          <stringProp name="HTTPSamplerProxy.contentEncoding"></stringProp>
          <stringProp name="HTTPSamplerProxy.path">/users/123</stringProp>
          <stringProp name="HTTPSamplerProxy.method">GET</stringProp>
          <boolProp name="HTTPSamplerProxy.follow_redirects">true</boolProp>
          <boolProp name="HTTPSamplerProxy.use_keepalive">true</boolProp>
          <boolProp name="HTTPSamplerProxy.do_ூ_multipart">false</boolProp>
          <boolProp name="HTTPSamplerProxy.image_parser">false</boolProp>
          <boolProp name="HTTPSamplerProxy.concurrentDownload">false</boolProp>
          <stringProp name="HTTPSamplerProxy.connect_timeout"></stringProp>
          <stringProp name="HTTPSamplerProxy.response_timeout"></stringProp>
        </HTTPSamplerProxy>
        <hashTree/>
        <ResultCollector guiclass="SummaryReportGui" testclass="ResultCollector" testname="Summary Report" enabled="true">
          <boolProp name="isSaveAsBinary">false</boolProp>
          <stringProp name="filename"></stringProp>
        </ResultCollector>
        <hashTree/>
        <ResultCollector guiclass="ViewResultsTreeGui" testclass="ResultCollector" testname="View Results Tree" enabled="true">
          <boolProp name="isSaveAsBinary">false</boolProp>
          <stringProp name="filename"></stringProp>
        </ResultCollector>
        <hashTree/>
      </hashTree>
    </hashTree>
  </hashTree>
</jmeterTestPlan>

This plan simulates 10 users (num_threads) making one request (loops: 1) to http://localhost:8080/users/123 with a 1-second ramp-up (ramp_time). We’re using a SummaryReport to see aggregate metrics and ViewResultsTree for detailed request/response data.

The core problem JMeter baseline tests solve is drift. Over time, due to code changes, infrastructure updates, or even subtle configuration tweaks, your application’s performance characteristics change. Without a baseline, you can’t reliably detect this drift. A baseline test establishes a "known good" state. You run it when your system is performing as expected, capture the metrics, and then use that snapshot as your reference point for future tests.

Here’s how it works internally: JMeter’s ThreadGroup simulates concurrent users. Each thread executes the samplers defined within it. The HTTPSamplerProxy makes HTTP requests. The ResultCollectors gather data about each request (response time, errors, bytes transferred, etc.). The SummaryReport aggregates these results, providing key metrics like Average, Median, 90% Line, and Error Rate.

The key levers you control are:

  • Number of Threads (num_threads): How many virtual users are hitting your system.
  • Ramp-up Period (ramp_time): How long it takes for all threads to start. A shorter ramp-up puts more immediate load on the system.
  • Loop Count (loops): How many times each thread will execute the samplers. For a baseline, you usually want enough loops to get statistically significant data but not so many that it becomes a full-blown load test. Often, 100-1000 loops per thread is a good starting point.
  • Duration (duration): An alternative to loop count, allowing the test to run for a specific time.
  • Sampler Configuration: The actual requests being made (URL, method, headers, body).
  • Assertions: To ensure the response is correct (e.g., ResponseAssertion for status codes or content).

When you run this, you’ll see something like this in the Summary Report:

---------------------------------------------------------------------
      1921044 in aggregate across all threads / Sampler: GET /users/{id}
      ...
      Average:        25 ms
      Median:         22 ms
      90% Line:       35 ms
      Min:            10 ms
      Max:            90 ms
      Error:          0.00 %
      Throughput:     398.0 /sec
      Received KBytes/sec:    45.34
      Sent KBytes/sec:        2.26
      Average Bytes:  114
---------------------------------------------------------------------

This report is your baseline. If next week, after a deployment, the "Average" jumps to 150ms or the "Error" percentage is no longer 0.00%, you know something has changed.

The real power of baseline tests comes not from the test plan itself, but from the process. You should version control your baseline test plans and run them regularly (e.g., daily, weekly, or before/after deployments). You also need a mechanism to store and compare the results over time. JMeter’s HTML Report (generated with jmeter -n -t your_test.jmx -l results.jtl -e -o report_dir) is excellent for this, providing trend graphs that are invaluable for spotting gradual performance degradation.

Most people treat JMeter results as a one-off snapshot. They run a test, look at the numbers, and then forget about it. The critical insight is that JMeter’s value in performance testing is its ability to track changes relative to a stable, version-controlled reference point. Without that, your "performance testing" is just a series of disconnected measurements.

The next logical step after establishing a baseline is to introduce controlled failures to understand your system’s breaking points.

Want structured learning?

Take the full Jmeter course →