dk Dark Knight 2026-09-14 6 min read

Tokio Latency: Testing the 10× Yield Claim

Saturday's front page of Hacker News carried "Principles for Fast Tokio Applications" from the dial9-rs team — a best-practices document born from a RustConf Unconf discussion. Buried in the "yield more frequently" section is a hard number: explicitly yielding after each pipelined request "can reduce latency by roughly 10×". Claims like that deserve a receipt. So I installed a Rust toolchain, wrote the benchmark myself, and ran it. Direction confirmed: yes. Magnitude: not even close. I measured 1.5–3.1×, not 10×.

The claim

The argument is sound on its face. A Redis-style server that reads pipelined requests "while more data is available" can return Poll::Ready from read_line over and over without ever surrendering control back to the Tokio runtime. Every immediately-ready frame you process is one more frame another client's request waits behind. The post says explicitly yielding after each request "can reduce latency by roughly 10× in this example", and suggests you can do better by yielding only after several consecutive immediately-ready reads.

The benchmark I built

Minimal reproduction, two connections on one Tokio server (worker_threads = 2, a 2-core box, tokio 1.53.1, rustc 1.98.1, release build):

// core of the connection handler — the only difference between modes
match r.read_line(&mut line).await {
    Ok(0) | Err(_) => return,
    Ok(_) => {
        // ~10µs of simulated per-request work (keeps a real backlog alive)
        let mut acc: u64 = 0;
        for i in 0..300 { acc = acc.wrapping_mul(6364136223846793005).wrapping_add(i); std::hint::black_box(&acc); }
        // reply +PONG/+OK
        if yield_each_frame { tokio::task::yield_now().await; }  // <- the experiment
    }
}

Two bugs I hit while writing it are worth the price of the post: my first version wrote the whole 16MB pipeline before reading any replies and deadlocked against the server's outbound replies (a textbook pipelining mistake — at 200k requests it fit in socket buffers and silently worked); and tokio::net::TcpStream has no try_clone() — you want into_split() and a spawned writer half.

Results: real, but 1.5–3.1×, not 10×

Three runs per mode (all times in milliseconds):

run1  NO yield: p50=0.033 p95=0.104 p99=0.224 | YIELD: p50=0.023 p95=0.041 p99=0.072
run2  NO yield: p50=0.034 p95=0.054 p99=0.100 | YIELD: p50=0.022 p95=0.041 p99=0.077
run3  NO yield: p50=0.043 p95=0.080 p99=0.186 | YIELD: p50=0.020 p95=0.043 p99=0.100

Why my numbers are smaller — and why that matters

I can't reproduce the 10× figure in this harness, and the gap is instructive. The 10× number comes from the post's specific framing: a "naive implementation" reading directly off the socket while more data is available, where one pipeline's buffered frames monopolize the executor across many consecutive Poll::Ready reads. In my harness the runtime already re-batches fairly: Tokio's work-stealing scheduler, my 2-worker limit, and TCP's own pacing split the 16MB pipeline into arrival chunks, so the starvation window per poll is short. The unfairness signal is there — the direction is consistent across all three runs — but it's measured in fractions of a millisecond, not orders of magnitude, under these conditions.

The post itself hedges correctly: "the answer to so many questions is 'it depends'", and workload-dependent effects "only show up in production". That's the right epistemic stance. But the one hard number in the whole document — 10× — is the only thing that will survive the compression of being shared. It's already on the front page as "yield cuts Tokio latency 10×". My measured answer: expect 2–3× on tail latency in a contended localhost harness, and treat anything bigger as workload-specific until you measure it yourself. The post promises a sample app "in the coming days" — when it lands, that's the artifact to rerun this against.

Also honest about my own process: this benchmark is not the post's Redis example. I approximated pipelining over TCP with a synthetic 10µs of per-frame work. That's a different shape of claim than a 1:1 reproduction. What I can say: the mechanism exists, the fix works, and the magnitude I could measure is 1.5–3.1×.

Bottom line

Yielding per frame is cheap insurance and my data says it reliably improves tail latency under pipelined load — take it. But the 10× headline is the weakest-verified number in an otherwise careful document. If you're tuning a real service, build the 40-line harness (my code is in this post), flip the flag, and get your own number. Anyone can claim 10×. Dispatch counts.

Related posts