rust: the self-contained flat word codec — 594% to 444% of generated C/C++

[gafferongames]

Campaign item 1 of the laggard round (#170): the Rust emitter learns the self-contained flat word-codec form. Wire bytes never move; checks=always is preserved exactly; nothing under bench/LOCK's prefixes is touched.

The move

Blended nine-language quick table, one caffeinated sitting, M2, generated subject:

lang	family	checks	before ns/msg	after ns/msg	before %	after %
cpp	gen	removed	1.72	1.70	100%	100%
c	gen	removed	1.85	1.86	107%	109%
java	gen	contract	6.49	6.50	377%	383%
rust	gen	always	10.22	7.53	594%	444%
cs	gen	always	13.59	13.53	790%	797%
dart	gen	contract	16.37	16.09	952%	949%
go	gen	always	26.93	26.55	1566%	1565%
js	gen	contract	75.53	74.69	4391%	4402%
elixir	gen	contract	288.68	287.10	16782%	16922%

Before column: the equalized sitting (bench/results/2026-08-31-equalized-quick-*). After: bench/results/2026-08-31-rustcodec-quick-*. The frozen reference re-ran unchanged — cpp gen within 2.4% write / 0.5% read, c within 0.2% / 0.7% — and every leg I did not touch landed within 2% of its equalized number, so the sitting is comparable and the movement is the emitter's. Rust is now the second-fastest generated leg.

Full rust leg, per shape (family gen, median msgs/s, M2):

shape	write before	write after		read before	read after
rigidbody_moving	50.1	50.8	1.01x	42.6	44.5	1.04x
rigidbody_at_rest	81.3	81.7	1.01x	75.1	78.5	1.05x
chat	67.1	68.8	1.03x	102.2	103.3	1.01x
test	224.7	876.4	3.90x	234.5	278.5	1.19x
inputpacket	47.0	70.7	1.51x	40.4	73.0	1.81x
shipcreate	77.3	84.9	1.10x	42.2	91.4	2.17x
probe_header	214.4	249.3	1.16x	153.5	182.8	1.19x
probebits	122.0	134.7	1.10x	134.0	149.7	1.12x
probearray	53.2	62.6	1.18x	63.0	66.1	1.05x
testdata	25.1	26.5	1.06x	21.7	24.3	1.12x
real_packet	8.6	10.0	1.16x	7.3	9.8	1.35x
bench_packet	65.9	70.8	1.07x	76.3	87.6	1.15x
bench_ints	108.2	132.8	1.23x	115.7	146.3	1.26x
bench_bits	117.0	148.4	1.27x	108.1	159.2	1.47x
bench_mixed	96.5	126.1	1.31x	102.1	136.0	1.33x
geomean			1.25x			1.26x

The hand-written rt/bits control rows — the same source before and after — moved 1.014x across ten rows. That is the negative control on the sitting: the generated rows moved because the generated code changed.

What the form is

Every bit offset inside a message is a generation-time constant no matter where the message starts. The old form did not use that: it handed the runtime one field at a time, which folds to nothing unless the spine lands inlined in a caller whose stream was just constructed. Entered with an unknown bit position — the regime every generated Rust call actually runs in — each serialize_bits reloads the packer's scratch, shifts by a runtime amount, tests a runtime carry and stores the state back. An eleven-field write paid eleven memory round trips through the stream.

internal/codegen/rust/flat.go folds the placement itself: field values into locals, OR'd into 32-bit chunk locals at literal shifts, one whole chunk per stream call. Reads fuse their bounds checks the same way — one per chunk instead of one per field — the technique the Java, Dart and JS-flat backends already carry.

let f0: u64 = u64::from(value.sequence as u32);
let f1: u64 = u64::from(value.ack_bits);
...
let mut w0 = (f0 | (f1 << 16)) as u32;
stream.serialize_bits(&mut w0, 32)?;
let mut w1 = ((f1 >> 16) | (f2 << 16) | (f3 << 28)) as u32;
stream.serialize_bits(&mut w1, 32)?;

Runs break at align, string/bytes, arrays, branches, nested struct and union calls, and the fixed-point / compressed-float / 128-bit families, whose value arithmetic lives in the runtime. Those keep the per-field form and start a fresh run after themselves. Runs also close at 256 bits so a hundred-field message does not hold every field value live at once. Chunk widths sum to the run's bit count exactly — the last chunk carries B mod 32 — so the flat form touches precisely the bits the per-field form did.

The public surface is unchanged: same function names, same (&mut WriteStream, &T) -> Result signatures, same types.

Wire and semantics

• Every wire golden unmoved (git status testdata/wire clean across the change).
• make test green: the full cross-language conformance suite, test/rust and test/rust-ludicrous included, plus the hostile-stream cases that pin specific error kinds (nonzero reserved → Validation, truncation → the stream's own error, out-of-range count → Error::Stream).
• corpus_id unchanged in every bench run: 457b96dfc5a0ffcd quick, bc776c004485e408 full.
• checks=always preserved: the same values are refused, with the same error, on every path. The checks-mode ruling stays the owner's.

Negative control, run and pasted:

$ printf '\xff' | dd of=testdata/wire/bench_mixed.bin bs=1 seek=3 count=1 conv=notrunc
$ cargo run --release -- --quick
WIRE GOLDEN MISMATCH: bench_mixed (21 golden vs 21 actual bytes) — refusing to bench code that does not match the corpus
BENCH FAILED (corpus_id 762ddd88ac9c941f)
$ # restore
$ cargo run --release -- --quick
bench_mixed        write     120.08 M msg/s ...
OK (corpus_id 457b96dfc5a0ffcd)

Two behaviours move, both named rather than smoothed:

1. Refusal timing on write. A run's refusals all run before the run packs, so a refused write has emitted fewer bits than before. Which values are refused, and with which error, is identical.
2. Error kind on a doubly-bad read. Fused bounds checks read a whole chunk at a time, so a stream that is both truncated inside a run and carries an out-of-range value before the truncation now surfaces Overflow where the per-field form surfaced the range refusal. Both refuse the packet; the set of accepted streams is unchanged, which is the property that matters at the trust boundary. This is the same consequence the Java/Dart/JS-flat fusion already carries.

Where the remaining 4.4x lives — and it is not the codec

Honest account: 444% is not C-tier, and I can name the whole residue.

The generated Rust spine is never inlined into the timed loop, and the emitter cannot fix that. #[inline(always)] on the spine is honoured — into the Fn::call shim standing between the bench driver and the spine. LLVM then prices that shim against its caller and refuses it. clang honours always_inline unconditionally, so the C and C++ legs never see this regime. Three measurements pin it:

• Flipping the read spine to #[inline(always)] in this harness moves bench_mixed by less than half a percent (97.16/102.97 vs 97.11/103.12). The attribute never reaches the loop.
• Extracting the timed loops into their own small functions, and passing closures instead of fn items, changed nothing. It is the cost model, not the call shape.
• Rebuilding the same binary with RUSTFLAGS="-C llvm-args=--inline-threshold=5000" — a diagnostic, not a shipped flag — moves the generated rows a further 2.28x geomean and the hand-written rt rows 4.54x. bench_mixed goes to 367 M/s write and ~444 M/s read ; rigidbody_moving read 10.0x; probe_header write 1034 M/s. Against generated C++'s 610/547 on the same sitting, that is 60% and ~81% — read approaches within ~1.23x and write is within 1.7x. (CORRECTED at review: this bullet originally claimed 606 M/s read and "read reaches parity" — the 606 was the hand-written rt row misattributed as the generated row; the reviewer's two independent runs measured gen read 444.5/437.8 M/s at 0.13% spread, and this PR's own later "450 M/s flat" sentence agrees. Read does NOT reach parity under the diagnostic.)

So the ceiling this form approaches, once the inlining discipline matches the reference legs, is near-C-tier on reads (~1.2x). The residue is LLVM's inliner cost model at the harness boundary, not language physics — and, per the reviewer's own measurement, not linkage in the LTO sense: plain idiomatic lto = "fat" + codegen-units = 1 (never measured by this PR; the repo has no [profile.release] at all) buys ~7% on read and nothing on write, nowhere near the threshold diagnostic's ~3x. That strengthens the cost-model diagnosis and retires the "linkage asymmetry" framing this paragraph first used. I did not equalize it here: it changes the measured leg's build or driver mid-round for one language only, and that is the owner's call, not a worker's. bench/rust/README.md claimed the driver "inlines like the C++ template reference"; that claim is now corrected to what is measured, with the diagnostic named.

The flat form is what makes the out-of-line regime cheap, which is why it lands regardless of how the inlining item is ruled — and it wins in the inlined regime too on reads (bench_mixed read 450 M/s flat vs 573 M/s per-field is the one row where the per-field form is ahead once inlined; write is 346 vs 300 the other way).

Aside, not shipped, as invited by the brief: stripping only the write-side range refusals — what a contract checks mode would do — buys +13.2% on bench_mixed write (126.1 → 142.7 M/s), reads unaffected. That is roughly 6% blended: 444% → ~420%. It does not change the picture, and the mode stays always.

Coverage gaps, named as follow-ons

• Compressed floats and fixed-point break every run — which is why rigidbody_* moved only 1.01–1.05x and examples128's generated Rust is byte-identical before and after. Both are pure functions of the value plus generation-time constants; folding their arithmetic into the run is the obvious next pass, gated on a differential against the runtime's own quantization.
• Arrays break runs. A fixed array of scalars is a static run of N * w bits; unrolling small ones into the enclosing run is a bounded, measurable next step (probearray 1.18x/1.05x is the row that would move).
• Nested struct and union calls break runs, so a message built from small structs flattens per struct rather than whole.

Land and expand: each is a separate measured pass, not a reason to hold this one.

🤖 Generated with Claude Code

[gafferongames]

Adversarial review verdict: MERGE, with one required correction now applied to the body. The reviewer reproduced everything on a fresh clone: zero test edits behind the behaviour movements (the hostile-stream error-kind pins unchanged and green; SPEC §5 mandates failure, not kind; the Java/Dart fusion precedent verified against those emitters — Rust's fusion is finer than Java's), zero-diff regen by its own rebuild, lock clean off the raw file list, all 30 per-shape cells and the blended table recomputed exactly, negative control 1.0142x over ten rows, and a code audit clearing precedence/shift-width/chunk-reuse/run-symmetry hazards with write_probe_bits hand-verified bit-for-bit.

THE CORRECTION: the diagnostic-build bullet claimed generated read 606 M/s / 111% of generated C++ / read parity. The 606 was the hand-written rt row misattributed; two independent reviewer runs measured gen read ~444 M/s at 0.13% spread (the PR's own later '450 M/s flat' sentence agrees). True figures: 60% write, ~81% read. The cost-model diagnosis survives and is STRENGTHENED — the reviewer also measured plain lto=fat + codegen-units=1 (never measured by the PR; no [profile.release] exists) at ~7% read / nothing write, so the barrier is the inliner cost model, not linkage.

Follow-ons filed on #170: name the blended table's statistic (max) beside the per-shape table's median per the doctrine; extend the fuzz compile leg to rustc (the flat form's compilability now rests on the fixed corpus — a wider class of layout-dependent spellings than the old uniform calls).