iyl.

iyi get github.com/sdogruyol/iyl

iyl

HTTP load testing that tells the truth, built for CI. One native binary, written in iyi: it drives every core, measures latency the way a user feels it, and turns a run into an exit code a pipeline can act on.

$ iyl http://localhost:3000/api -d 30s -c 64 --max-p99 250ms

• Honest latency. Under --rate, every request is also timed from the slot the schedule gave it, so a target that stalls is charged for the queue it caused instead of looking faster (below).
• Gates, not graphs. --max-p99, --min-rps, --max-fail-rate and the rest exit 1 when breached; --url-threshold scopes a gate to some URLs; --baseline fails a run that regressed against an earlier one.
• Exit codes that mean something. A slow service (1), a broken command line (2) and a target that never answered (3) are different failures.
• Where the time went. DNS, connect and TLS per new connection; time to first byte and total per response; per status, per URL, per error kind.
• Reports for machines and people. Text with color, JSON with a versioned schema, CSV one row per run, and Markdown for a job summary or a pull request. --save keeps the JSON beside any of them.
• Multi-core. Connections are spread over worker threads, each worker's connections taking turns on it, nothing shared but a handful of atomics.
• HTTP/1.1 done carefully. Keep-alive, bodies framed by length, chunks or close, read through one fixed buffer per connection and never kept; redirects, proxies (CONNECT for https), Basic and Bearer credentials, cookies, custom success codes, request deadlines.

Install

iyl is built from source with the iyi toolchain, 0.21.0 or later (install iyi):

git clone https://github.com/sdogruyol/iyl && cd iyl
mkdir -p bin && iyi build --release -o bin/iyl iyl.iyi
bin/iyl --version

The result is a single executable with no runtime to install and no OpenSSL: TLS is iyi's own.

Usage

$ iyl [options] <url>...

# 10,000 requests over 100 connections
iyl http://localhost:3000 -n 10000 -c 100

# 30 seconds at a steady 500 requests a second, gated on the honest p99
iyl http://localhost:3000/api -d 30s -q 500 --max-p99 250 --fail-on-rate-miss

# POST JSON, warm up first, never wait more than 2s on one request
iyl http://localhost:3000/orders -d 1m -c 32 -m POST \
  --content-type application/json -b '{"sku":"A-1","qty":2}' \
  --warmup 5s --request-timeout 2s

# Several endpoints in turn, each with its own gate
iyl --urls-file urls.txt -d 30s -c 50 \
  --url-threshold "/search max-p99 400" \
  --url-threshold "/orders max-fail-rate 0.5" \
  --min-rps 2000

iyl --help lists every option:

A URL without a scheme is http://; credentials written in a URL (http://user:pass@host/) are sent as Basic.

Honest latency

A load tester that measures from the moment it sends a request has a blind spot exactly where it matters. When the target stalls, the connections that would have sent requests are stuck waiting, nothing is sent, and so nothing slow is measured: the report shows the fast answers from before and after the stall. A target that stops answering for a second looks better. This is coordinated omission.

Give iyl a rate and it keeps a schedule: request k is due at start + k / rate, whatever happened before it. Slots are handed out on that fixed timeline, never re-anchored to "now", so a stall leaves a backlog rather than a gap, and each request is timed twice:

• service — from when it was actually sent to the last byte of its answer: what the server spen
• corrected — from when it was due to the last byte: what a user who arrived on schedule waite

Their difference is the send delay, reported on its own. The schedule also says how many requests fell due and how many were sent; the rest are skipped, so a run can never claim a rate it did not reach.

Here is a target that takes 200 ms per request, asked for 50 a second on one connection:

Latency           avg      p50      p90      p95      p99    p99.9      max
  service      200.8ms  201.6ms  201.6ms  203.6ms  203.6ms  203.6ms  203.7ms
  corrected      1.83s    1.84s    3.27s    3.46s    3.46s    3.46s    3.46s
  send delay     1.63s    1.63s    3.08s    3.26s    3.26s    3.26s    3.26s

Schedule
  requested 50/s · attained 5.0/s (10.0%)
  scheduled 150 · issued 20 · skipped 130 · worst send delay 3.26s

Gates
  ✘ max-p99                  corrected_p99 3.46s <= 300.0ms
  ✘ fail-on-rate-miss        attainment_percent 9.95 >= 99.00

The service p99 of 204 ms is true and would have passed a 300 ms gate; it is also not what anyone experienced. Latency gates read the corrected numbers whenever --rate is set; --latency-correction off makes them read service time. Without --rate there is no schedule, and the two are the same.

In CI

Gates

Any breached gate exits 1. Latency limits are milliseconds, or a time with its unit (1.5s).

A --url-threshold whose pattern matches none of the URLs under load is a usage error, so a renamed endpoint fails the build instead of quietly losing its gate.

Exit codes

A breached gate outranks a signal; an unreachable target outranks a gate, since there was nothing to measure. The JSON report carries the same as verdict.exit_code and verdict.reason.

GitHub Actions

- name: Load test
  run: |
    iyl http://localhost:3000/api -d 30s -q 500 \
      --max-p99 250 --fail-on-rate-miss \
      --baseline perf/baseline.json --save result.json \
      -o markdown >> "$GITHUB_STEP_SUMMARY"

The Markdown report lands in the job's summary; result.json is the next run's baseline.

Reports

--json prints one document; every key is always present, null where it does not apply. All latencies are milliseconds.

Transport errors are one of a fixed vocabulary, so a pipeline can match on them: dns_failure, connect_refused, connect_timeout, connect_failed, read_timeout, write_timeout, request_timeout, connection_reset, connection_closed, tls_error, proxy_error, protocol_error, other. A request whose kept connection the server had closed while idle is sent again once on a new one, when its method is idempotent, and is not counted as a failure.

-o csv writes a header and one row: the summary and the latency percentiles, ready to be appended to a history file.

How it works

Each worker is a task on iyi's pool of threads (spawn_parallel), and each of its connections is a task on that worker, taking turns at every wait on the network. What all workers read — the plan, built once from the command line — is immutable, which the compiler checks (Share, SPEC.md III.4.4): the only things they write together are the atomics behind the schedule, the request count and the progress line. Each worker records into its own histograms without a lock; they are merged once, at the

Latencies go into log-bucketed histograms, 1% wide from a microsecond to an hour, with Welford's mean and variance, so percentiles cost no samples and workers merge exactly. A response is parsed where it lies in the connection's buffer: status, framing and the few headers that matter are read off the bytes, and the body is counted on its way past.

TLS is iyi's own (TLS 1.3, and 1.2 for a server that speaks no newer). A server's certificate is always checked: against the system's roots, or the CA certificates --cacert names. There is deliberately no switch to skip the check; a test server's CA goes in --cacert instead.

Performance

iyl http://127.0.0.1:3901/ -d 3s -c 128 -w N against iyi's own HTTP server on the same machine (Windows 11, 12 virtual CPUs under QEMU, loopback, both built --release, 2-byte answers), the median of three runs:

One worker reaches 96% of what a bare loop of socket writes and reads, with no parsing and no recording, reaches on the same machine and connections (19,266 a second): measuring costs little. Past 8 workers the client and the target fight for the same 12 cores and throughput stops rising: -w is a dial, and a number worth reading is one taken where raising it no longer helps.

These are loopback numbers on one virtual machine, a ceiling for the load generator rather than a claim about any network.

Development

iyi test                                         # every test/*_test.iyi
iyi run test/support/target_server.iyi -- 3000   # a target to point iyl at
mkdir -p bin && iyi build -o bin/iyl iyl.iyi && bin/iyl http://127.0.0.1:3000/bytes/1024 -d 5s

The end-to-end tests run iyl against servers in their own process: plain, TLS under a test CA (test/fixtures), and a proxy that tunnels.

License

MIT