Writing realistic load tests with k6: open vs closed models, arrival rates and coordinated omission

Most load tests answer a slightly different question from the one the team actually asked. The team wants to know: what happens to our users when 3× the normal traffic arrives? The script measures: what happens when 200 robots hammer the API as fast as it lets them? These sound similar. Once the system starts to slow down, they give very different answers.

This article goes through the decisions that make a k6 test realistic: the workload model, the executor, thresholds, think time and correlation. It also covers the measurement trap called coordinated omission. All examples use the k6 JavaScript API and run as shown.

Closed vs open workload models

A load generator follows one of two models, whether or not you chose it deliberately.

In a closed model, a fixed number of virtual users (VUs) loop. Each VU sends a request, waits for the response, optionally sleeps, and repeats. Throughput depends on response time: with 50 VUs, 1 second of sleep and 100 ms responses, you get about 50 / 1.1 ≈ 45 requests per second. If responses slow to 2 seconds, the same 50 VUs only produce 50 / 3 ≈ 17 requests per second. The load drops exactly when the system is struggling.

In an open model, new work arrives on a schedule that doesn't depend on the system. Ten new users per second keep arriving whether the last ten were served in 100 ms or 10 seconds. That is how real traffic to most public APIs and web apps behaves: your customers don't know the checkout is slow and politely wait before clicking.

In k6, constant-vus, ramping-vus, per-vu-iterations and shared-iterations are closed-model executors. constant-arrival-rate and ramping-arrival-rate are open-model executors. Here is a typical closed-model script:

import http from 'k6/http';
import { sleep } from 'k6';

// Closed model: a fixed pool of 50 VUs. Each VU sends the next request
// only after the previous one has returned (plus 1 s of sleep).
export const options = {
  scenarios: {
    closed: { executor: 'constant-vus', vus: 50, duration: '10m' },
  },
};

export default function () {
  http.get(`${__ENV.BASE_URL}/v1/products`);
  sleep(1);
}

It isn't wrong, but it measures something specific: the behaviour of a system serving a fixed population that backs off when things slow down. That fits some systems, such as a fixed pool of back-office workers or a batch consumer with a fixed number of threads. For a public API facing internet traffic, it is usually the wrong model.

Arrival-rate executors in practice

Arrival-rate executors schedule iterations, not VUs. You state how many iterations should start per timeUnit, and k6 takes a free VU from a pool to run each one. If no VU is free, k6 allocates more, up to maxVUs. If it still can't start an iteration on time, it skips it and increments the dropped_iterations metric.

import http from 'k6/http';
import { check } from 'k6';

const BASE = __ENV.BASE_URL;

// Open model: k6 starts new iterations at a fixed schedule,
// regardless of how fast the system under test responds.
export const options = {
  scenarios: {
    create_orders: {
      executor: 'ramping-arrival-rate',
      startRate: 10,          // iterations per timeUnit at t = 0
      timeUnit: '1s',
      preAllocatedVUs: 100,   // initialised before the test starts
      maxVUs: 1000,           // ceiling; beyond it, iterations are dropped
      stages: [
        { target: 50, duration: '5m' },    // ramp to normal peak (50 orders/s)
        { target: 50, duration: '15m' },   // hold
        { target: 150, duration: '5m' },   // ramp to 3x peak
        { target: 150, duration: '10m' },  // hold
        { target: 0, duration: '2m' },     // ramp down
      ],
    },
  },
  thresholds: {
    'http_req_duration{name:create_order}': ['p(95)<500', 'p(99)<1200'],
    http_req_failed: ['rate<0.01'],
    checks: ['rate>0.99'],
    // Any dropped iteration means the schedule was not kept: treat as a failed run.
    dropped_iterations: ['count==0'],
  },
};

export default function () {
  const res = http.post(
    `${BASE}/v1/orders`,
    JSON.stringify({ sku: 'SKU-1001', qty: 1 }),
    { headers: { 'Content-Type': 'application/json' }, tags: { name: 'create_order' } },
  );
  check(res, { 'status is 201': (r) => r.status === 201 });
}

A few details matter here:

• preAllocatedVUs are initialised before the test starts. VUs allocated beyond that happen mid-test, and initialising a VU costs CPU and memory on the load generator. Pre-allocate close to what you expect to need.
• maxVUs is a safety ceiling, not a target. If the test reaches it, either the system has become very slow (each iteration holds a VU longer) or the ceiling is too low.
• dropped_iterations is the most important signal in an open-model test. A non-zero count means k6 couldn't keep the schedule, so the offered load was lower than you think. The threshold count==0 turns that into a hard failure.

How many VUs do you need? Use Little's Law: concurrency = arrival rate × time in system. At 50 iterations per second with a mean iteration duration of 0.4 s, you need about 20 busy VUs on average. If latency degrades to 4 s under stress, you need 200. Size preAllocatedVUs for normal conditions and maxVUs for the degraded case you want to observe.

Building the workload model from real traffic

An executor is only as good as the numbers you give it. Before writing the script, take a week of access logs or APM data and extract:

• Peak arrival rate per endpoint: not the daily average, but the busiest minute or five minutes, and when it happens (payday, Monday morning, month-end batch).
• The request mix: if 70% of traffic is reads and 2% is payment creation, a test that only hammers payments is a stress test of one endpoint, not a load test of the system.
• Sessions: how many requests a typical user journey makes and how long people pause between steps.
• Data shape: how many accounts, products or tenants are active. Caches behave very differently when every request hits the same record.

In k6, model different traffic classes as separate scenarios, each with its own executor, rate and exec function. A typical mix might use one scenario for anonymous browsing at 200 iterations/s, one for logged-in journeys at 20/s and one for webhooks from a partner at 5/s. Thresholds can then target each scenario through the automatic scenario tag.

Thresholds that mean something

Thresholds turn a load test from a chart into a pass/fail check that can run in CI. Three rules help:

1. Threshold on percentiles, per endpoint. A global p(95) across all requests mixes 5 ms health checks with 800 ms report generation and hides both. Tag requests with a stable name and set thresholds on http_req_duration{name:create_order}. For URLs with IDs in the path, either set tags.name yourself or use the http.url tagged template literal, which groups them automatically.
2. Derive the numbers from SLOs, not from the last run. If the product promise is "payments confirm in under a second for 99% of users", the threshold is p(99)<1000 on that endpoint. A threshold taken from yesterday's result only tells you whether anything changed.
3. Always pair latency with errors. A system that answers every request with a fast 503 has excellent latency. Add http_req_failed and, for business correctness, a rate threshold on checks.

Use abortOnFail carefully. It saves time in CI smoke tests. In a capacity test, though, the most valuable data usually comes after the first threshold breaks.

Think time and realistic journeys

Think time is the pause between a user's actions. In a closed model, it effectively controls the request rate, which is why closed-model scripts so often end with sleep(1). In an open model, the arrival rate controls how many journeys start, and think time controls how long each journey lasts. That changes concurrency, the number of sessions in flight at once, which in turn affects session stores, connection pools and caches.

So use think time to model real gaps inside a journey (reading an account page before starting a transfer), randomise it, and don't add a sleep at the end of an arrival-rate iteration "to pace it". Pacing is the executor's job, and a trailing sleep just holds a VU for nothing.

Correlation and test data

Real users don't send the same hard-coded request. They log in, get a token, list their accounts and act on IDs the server returned. A script that skips this makes every VU hit the same cached record with the same token. The test then measures your cache, not your system.

import http from 'k6/http';
import { check, sleep } from 'k6';
import { SharedArray } from 'k6/data';
import exec from 'k6/execution';

const BASE = __ENV.BASE_URL;
const RUN_ID = __ENV.RUN_ID || `${Date.now()}`;

// Loaded once in the init context and shared read-only between VUs.
// users.json: [{ "email": "...", "password": "...", "payee_id": "..." }, ...]
const users = new SharedArray('users', () => JSON.parse(open('./users.json')));

export const options = {
  scenarios: {
    banking_journey: {
      executor: 'constant-arrival-rate',
      rate: 20,              // 20 new journeys per second
      timeUnit: '1s',
      duration: '20m',
      preAllocatedVUs: 250,  // ~ rate x mean journey duration (see Little's Law)
      maxVUs: 800,
    },
  },
  thresholds: {
    'http_req_duration{name:login}': ['p(95)<300'],
    'http_req_duration{name:get_account}': ['p(95)<200'],
    'http_req_duration{name:create_transfer}': ['p(95)<600'],
    http_req_failed: ['rate<0.01'],
    dropped_iterations: ['count==0'],
  },
};

function params(name, token, extraHeaders) {
  const headers = Object.assign({ 'Content-Type': 'application/json' }, extraHeaders || {});
  if (token) headers.Authorization = `Bearer ${token}`;
  return { headers: headers, tags: { name: name } };
}

// Randomised think time between min and max seconds.
function think(min, max) {
  sleep(min + Math.random() * (max - min));
}

export default function () {
  // A different user per iteration, not per VU: spreads load across accounts.
  const user = users[exec.scenario.iterationInTest % users.length];

  const login = http.post(`${BASE}/v1/auth/login`,
    JSON.stringify({ email: user.email, password: user.password }), params('login'));
  if (!check(login, { 'login: 200': (r) => r.status === 200 })) {
    return; // no token: abandon this journey instead of producing junk 401s
  }
  const token = login.json('access_token');           // correlation #1

  think(2, 5);

  const list = http.get(`${BASE}/v1/accounts`, params('list_accounts', token));
  const accountId = list.json('data.0.id');            // correlation #2

  const account = http.get(`${BASE}/v1/accounts/${accountId}`, params('get_account', token));
  check(account, { 'account: 200': (r) => r.status === 200 });

  think(3, 8);

  const transfer = http.post(`${BASE}/v1/transfers`,
    JSON.stringify({ from: accountId, to: user.payee_id, amount: '10.00', currency: 'EUR' }),
    params('create_transfer', token,
      { 'Idempotency-Key': `${RUN_ID}-${exec.scenario.iterationInTest}` }));
  check(transfer, { 'transfer: 201': (r) => r.status === 201 });
}

What this script does right:

• Correlation: the token and account ID come from earlier responses (res.json() accepts a GJSON-style path such as data.0.id).
• Test data at scale: SharedArray loads the user file once and shares it read-only between VUs. Without it, every VU would hold its own copy in memory. Selecting by exec.scenario.iterationInTest spreads iterations across users instead of pinning one user to each VU.
• Failing fast inside a journey: if login fails, the iteration stops. Otherwise the following requests fail with 401s that inflate error counts and distort latency for those endpoints.
• Idempotency keys are unique per iteration and per run, so retries inside the system behave as they would in production and repeated runs don't collide.

Make sure the test data matches production cardinality. Five test users fronted by a cache are not five thousand real ones. Also plan how to reset state: a 20-minute run at 20 transfers per second creates 24,000 transfers.

Coordinated omission: the measurement trap

Gil Tene coined the term coordinated omission for a flaw in how many load tools measure latency. Picture a closed-model tester that sends one request per second. The server stalls for 10 seconds. The tester sends request #1, waits 10 seconds for the answer, and records one slow sample of 10 s. The nine requests that would have been sent during the stall were never sent, so they were never measured. If real users had kept arriving at one per second, they would have waited 9 s, 8 s, 7 s and so on. The tool "coordinated" with the system under test and omitted exactly the samples that mattered.

The result is percentiles that look far healthier than reality, especially p99 and above. During a stall, the gap between the reported p99 and the experienced p99 can be an order of magnitude.

How k6 helps, and where it doesn't:

• Arrival-rate executors keep starting iterations on schedule while free VUs exist, even if earlier iterations are stuck. Slow periods produce many slow samples, as they would with real users.
• k6 measures each request from when it is actually sent. If the executor runs out of VUs, it doesn't start the iteration late and pretend it was on time. It drops it and counts it in dropped_iterations. That's why that metric must be zero for a valid run: dropped iterations are omitted measurements.
• Closed-model executors are subject to coordinated omission by design. Use them only when a closed population is genuinely what you are modelling.

The load generator itself can also cause coordinated omission. If its CPU is saturated, requests leave late and timings get noisy. Monitor the generator's CPU and network as well, keep it well below saturation, and distribute the test across several machines if you need to.

A checklist before you trust the numbers

• The executor matches reality: arrival-rate for open traffic, VU-based only for genuinely closed populations.
• Rates and mix come from production data, including the real peak, not an average.
• dropped_iterations is zero, and the load generator's CPU stayed well below saturation.
• Thresholds are per endpoint, percentile-based, tied to SLOs and paired with error rates.
• Requests are correlated, test data has production-like cardinality, and caches are neither artificially hot nor artificially cold.
• The environment is documented (instance sizes, data volume, versions) so the run can be repeated and compared.
• Raw results are kept, not just the end-of-test summary:

# Run against staging, tag the run, and keep raw results for later analysis
k6 run \
  -e BASE_URL=https://staging.example.test \
  -e RUN_ID="$(date +%Y%m%d-%H%M)" \
  --summary-trend-stats="avg,min,med,p(90),p(95),p(99),p(99.9),max" \
  --out json=results.json \
  journey.js

The default end-of-test summary is a good start, but it aggregates the whole run into one set of numbers. A 2-minute latency spike in the middle of a 30-minute test barely moves the overall p95. Keep the raw time series (JSON output, or a stream to Prometheus or Grafana Cloud k6) so you can see when things went wrong, not just whether they did. Reading those results properly is a topic of its own, covered in our article on percentiles, histograms and saturation.