cs: the codec pass — union batch cores, scoped batches, and the flat word codec (write 1.59x, round_trip 1.60x; 361% -> 219% of generated C++) by gafferongames · Pull Request #208 · mas-bandwidth/schema

gafferongames · 2026-08-31T23:19:52Z

The C# backend was the laggard on the current ledger — 361% of generated C/C++ on bench/results/2026-09-01-arm64-macbook.csv (main at d00cf49) — and the only backend never audited for generation quality (#170's matrix). This is that audit and the levers it found.

Headline (--only cs, both halves committed, two sittings back to back, bench_mixed, family gen, checks=always):

path	before max	after max	before median	after median
write	2,104,563	3,342,814	2,103,454	3,334,630	1.59x
round_trip	992,155	1,594,245	990,865	1,586,844	1.60x

Against the same day's quiet-box reference at the same corpus_id (cpp round_trip median 3,467,551), on the round_trip statistic: 350% → 219% of generated C++, and on write 347% → 219%. Zero allocations on the hot path throughout (the leg's own assert). Wire bytes never moved: full make test green across nine backends on every commit, degenerate corpus included.

1. MEASURE FIRST — where the time actually was

Instrument: DOTNET_JitDisasm + DOTNET_JitDisasmSummary, DOTNET_TieredCompilation=0, FullOpts, against the built bench/cs Release binary (BENCH-STANDARD.md §4.1's own recipe). Not dotnet-trace: every hot call inlines, so a sampled stack would attribute the whole codec to one frame and say nothing.

Finding 1 — this is NOT Go's distribution. #198 found ~86% of Go's time inside runtime calls the compiler would not inline. C# has no such wall: ReadBenchMixed and WriteBenchMixed contain ZERO direct bl into serialize.cs. Every entry point the generated code touches inlines. Anyone porting #198's conclusions here would have optimised the wrong thing.

Finding 2 — the cost was the generated code's own composition. BenchMixed reaches a union, and the batch plan excluded any type that reached one, so the whole message ran on the stream and reached its element types through their STREAM entries — each of which opens a batch and closes it again:

site	elements	BeginBatch/End round trips	real calls
`entities`	8	8	8
`stats`	80	80	80
`game_event`	1	1	1 + 1 arm

89 capture/restore round trips per 438-byte message, on the two arrays carrying 72% of the wire bits.

Finding 3 — what the inlined bodies do. BitWriter's packer step masks, shifts, ORs into scratch, tests newScratchBits >= 64, conditionally stores a qword and recovers the spill: a data-dependent branch and a conditional store PER FIELD. MixedEntity paid fourteen for 135 bits.

Finding 4 — read-path code density. ReadBitsUnchecked's slack-free window-assembly fallback loop is inlined at EVERY field read site — a loop that never runs on a slack buffer. ReadBenchMixed was 8952 bytes of code for 869 bytes of IL.

2. THE LEVERS, ranked, and what each returned

#	lever	where	status
A	Union batch cores + a scoped batch per composition site — 89 capture/restore round trips become 3	emitter	LANDED — write +34%, round_trip +11%
A'	Whole-message batch, the obvious companion to A	emitter	REFUSED BY MEASUREMENT — write 2.10→2.04, round_trip 0.97→0.76 M msg/s. See §3.
B1	The flat word codec, write half — values into locals, OR'd into 64-bit chunk words at literal shifts	emitter	LANDED — write +17%, round_trip +9%
B2	The flat word codec, read half — one bounds check per chunk, one sticky-error test per run	emitter	LANDED — round_trip +30%
E	Public refusal-latch on `ReadStream`/`ReadBatch`, so B2 could fold the ranged reads it leaves on the runtime call	serialize.cs	RANKED AND REFUSED — worth ~1%. See §4.
C	Hoist the read slack-path out of line (`NoInlining` cold helper) so the fallback stops being pasted at every read site	serialize.cs	not taken — B2 cut the read sites from 14 to 3 per entity, which removes most of what made this attractive
D	Branch-free 64-bit chunk primitives on the batch (`value >> 1 >> (63 - sb)` handles `sb == 0` branchlessly)	serialize.cs	not taken — ranked behind C for the same reason
F	Group K array elements into one run (80 stats × 18 bits = 80 chunk touches today; K=8 would make it 30)	emitter	the next real lever — Go's `flatGroupRun`, not ported here (land-and-expand)
G	Unroll small fixed scalar arrays into runs (`loadout [4]uint8`)	emitter	small, not taken
H	The C# bench driver's write/read delegates (one indirect call per op cpp and rust do not pay)	bench/cs	NOT TAKEN — harness, and cross-leg comparability is not mine to move

3. Lever A' — the measured refusal, stated

Once unions carry cores, nothing stops the whole message body running on one batch. It is slower, and materially. The JIT DID inline the 12.7 KB core (entry IL size 23), so this is not the address-exposure law from the original batch work firing. It is the delegated sites: BenchMixed's body has six (fixed, two bytes runs, ufixed, uint128, int128), each a Sync down and a Recapture back, and between them a ref struct the register allocator must keep alive across 12.7 KB of code.

batchEntry now states it: density is necessary, b == 0 is the rest.

4. Lever E — refused on evidence, not deferred

B2 folds a ranged read into a run only where the range check is VACUOUS (max - min == 2^bits - 1), or where this emitter's read path already refuses without latching (the full-range unsigned path). Everything else keeps the runtime call, because SerializeInt LATCHES ValueOutOfRange and a generated comparison would not — cs publishes checks=always and test/cs pins that latch.

The obvious follow-on was a public refusal-latch on ReadStream/ReadBatch so the rest could fold. A hand-written prototype that folded every ranged read of the two hot cores, latch semantics and all, measured round_trip 1.55 M msg/s. The landed restriction measures 1.54. The API addition is worth about 1%. Recorded as refused.

5. Discipline

Wire bytes never move. Full make test green across nine backends on every commit — degenerate corpus, test/cs and test/cs-ludicrous (which pin the corrupt-wire refusals) included. The bench's own golden + per-variant round-trip gates pass at corpus_id 6b213fbfa1a03a99.
Negative control, run rather than assumed. With flatMaskLit forced to emit no mask — the flat form's masking invariant deliberately broken — make test produces 65 failure lines. The gate that proves this form does see it.
Checks semantics preserved exactly. Same inputs refused, same errors, on every path. The two consequences fusion always has (which error surfaces when a stream is both truncated and out of range; a failed run leaving the destination untouched) are named in flat.go, and are the ones Rust, Go, Java, Dart and JS-flat already carry.
Zero allocations on the hot path, still 0 bytes/pass.
One commit per lever, one measurement per commit, attribution never bundled.
Provenance: bench/run.sh's schema commit line now carries its branch, like every runtime line already did — a small fix for a failure mode this repo has had twice (a sweep that measured the wrong tree; a published result whose SHA was rebased away). Law-home in BENCH-STANDARD.md §3.5.
The bits family rows in the committed pair are flagged in the CSVs themselves as NOT attributable to this change: BitsBench.cs touches no generated code, yet its rows move 11% across the halves at sub-3% spread. Process-level, an open question, not a result.

🤖 Generated with Claude Code

The C# backend was the only one never audited for generation quality, and the profile of the bench shape found the cost in one place: composition. MEASURED FIRST (DOTNET_JitDisasm + DOTNET_JitDisasmSummary, FullOpts, tiering off, on generated/bench/cs/Bench.cs). Everything the runtime offers already inlines — zero direct calls into serialize.cs from any generated method. What did not inline was the generated code's own composition: the entity array (8 elements) and the stat array (80 elements) each reached their element type through its STREAM entry, and each of those entries opened a batch and closed it again. 88 BeginBatch captures and 88 End restores per 438-byte message, plus 88 real calls the JIT will not inline through, on a message where those two arrays carry 72% of the wire bits. They reached the stream entry because BenchMixed reaches a union, and the batch plan excluded any type that reached one — a union's arm dispatch had no core to compose with. That exclusion was landed as land-simple with the note "a follow-on if a profile ever convicts the plain path". It has. TWO CHANGES, one law each. 1. Unions carry batch cores. A union's arm dispatch is a switch over calls, and inside a core those calls go core-to-core by ref like any other composition — nothing about the inline-only rule ever stood in the way, only the missing emitter half. densityUnion weighs the tag plus the heaviest arm; the core closure walks arms as well as struct fields. 2. A composition site inside a stream body opens a SCOPED batch — one capture and one restore for the whole site, an array's entire loop included. Only the composition call rides inside the scope (an array's count site stays on the stream ahead of it), so the scope has exactly one way to fail and End always runs before the refusal leaves the function. AND ONE MEASURED REFUSAL, recorded because it is a result. The obvious third change — let the whole message body run on one batch now that the union no longer blocks it — is SLOWER, and materially: bench_mixed write 2.10 -> 2.04 M msg/s and round_trip 0.97 -> 0.76 M msg/s against the same baseline the two changes above improve. The JIT did inline the 12.7 KB core (IL size 23 at the entry), so this is not the address-exposure law firing; it is the delegated sites. A body with any bulk, 128-bit or fixed-point site must Sync the batch down and Recapture it around that site, and between them it hands the register allocator a ref struct that has to survive the whole body. batchEntry states it: density is necessary, b == 0 is the rest. MEASURED, this branch vs main, same box, bench --quick, median of 3 sittings (iteration instrument, not certification): bench_mixed write 2.10 -> 2.82 M msg/s (+34%) bench_mixed round_trip 0.97 -> 1.08 M msg/s (+11%) Wire bytes unmoved: full `make test` green across nine backends, the degenerate corpus included, and the bench's own golden + per-variant round-trip gates pass at corpus_id 6b213fbfa1a03a99. Checks semantics are untouched — the same inputs are refused with the same errors. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

The third port of the template Rust (#183) and Go (#198) already carry: field values into locals, OR'd into 64-bit chunk words at literal shifts, one packer step per chunk instead of one per field. WHY IT PAYS HERE IS NOT WHY IT PAID IN GO. #198's finding was that Go's per-field entry points sit past the compiler's inlining budget, so every field cost a real call and ~86% of the codec's time was inside them. C# has no such wall: the JitDisasm audit found ZERO direct calls into serialize.cs from any generated method — every entry point inlines. What C# pays instead is what those inlined bodies DO. BitWriter's packer step masks, shifts, ORs into the scratch word, tests `newScratchBits >= 64`, conditionally stores a qword and recovers the spill: a data-dependent branch and a conditional store PER FIELD. MixedEntity paid fourteen of them for 135 bits. It now pays three. THREE INVARIANTS, RE-DERIVED RATHER THAN INHERITED — #198's review record is why they are spelled out in flat.go's header rather than assumed: 1. 64-bit chunks are wire-identical because serialize.cs's SerializeBits64 splits low-dword-first-then-remainder, which IS the 32-bit chunk order. 2. Every piece is masked to its width, because BitWriter.WriteBitsUnchecked TRUNCATES a too-wide value rather than refusing it — the flat form must truncate identically or a wide value corrupts its neighbours in the chunk instead of losing its own high bits. This is the invariant the Go port lost by inheriting Rust's fallback; here it is enforced at the one place a piece's value expression is built. 3. Write refusals come first, in declaration order — same values refused, same non-latching false. What moves is how many bits reached the stream before the refusal, and a refused write leaves a partial message either way. A run breaks at align, strings/bytes, arrays, branches, nested struct and union calls, and the float, fixed-point, compressed-float and 128-bit families. A run that would not REDUCE the packer-step count is not flattened. A run that does not pay falls back ITEM BY ITEM — the item rides inside every piece for exactly that reason, because the item is the only unit the per-field path can re-emit. THE READ HALF IS NOT IN THIS COMMIT, and the blocker is named rather than worked around: folding a ranged read replaces SerializeInt with a generated comparison, and serialize.cs's ranged read LATCHES ValueOutOfRange where a generated guard returns false without latching. cs publishes checks=always and test/cs pins that latch, so the read fold waits on a public refusal-latch on ReadStream/ReadBatch — a serialize.cs change, its own PR. The write path has no such dependence: its folded range refusals already return false without latching. Measured ceiling for the read half, from a hand-written flat prototype of the two hot cores: round_trip 1.08 -> 1.55 M msg/s, a 42% cut in read time. MEASURED, against the previous commit, same box, median of 3 sittings: bench_mixed write 2.82 -> 3.30 M msg/s (+17%) bench_mixed round_trip 1.08 -> 1.18 M msg/s (+9%) and against main: bench_mixed write 2.10 -> 3.30 M msg/s (+57%) bench_mixed round_trip 0.97 -> 1.18 M msg/s (+22%) flatMaxRunBits is Go's 384 carried unmeasured: swept at 128/256/512 the C# numbers do not move, because this corpus's longest run is 135 bits and no cap in that range binds it. Said out loud in the file rather than presented as a C# measurement. Wire bytes unmoved: full `make test` green across nine backends, the degenerate corpus included, and the bench's golden + per-variant round-trip gates pass at corpus_id 6b213fbfa1a03a99. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

The read half of the same form: chunks read first, ONE sticky-error test for the whole run, then every value unpacked, validated and stored with literal shifts and literal masks. Where the per-field form paid a bounds check and an error test per field, MixedEntity now pays one of each per chunk. WHAT IT WILL NOT ABSORB, AND WHY. A ranged read whose refusal can actually fire goes through serialize.cs's SerializeInt/SerializeInt64, which LATCH SerializeError.ValueOutOfRange; a generated comparison returns false without latching. cs publishes checks=always and test/cs pins that latch, so a ranged piece joins a run ONLY where its check is VACUOUS — max - min is exactly 2^bits - 1, so no value the bits can hold is out of range — or where this emitter's own read path ALREADY refuses without latching (the full-range unsigned bits64 path, which has carried a generated refusal all along). Everything else closes the run and keeps the runtime call. Same inputs refused, same errors, on every path. AND THAT RESTRICTION COSTS ALMOST NOTHING — which retires a lever rather than deferring it. The hand-written prototype that folded EVERY ranged read of the two hot cores, latch semantics and all, measured round_trip 1.55 M msg/s. This commit, which leaves four of MixedEntity's fourteen fields on the runtime call because their range checks can fire, measures 1.54. The public refusal-latch on ReadStream/ReadBatch that the full fold would need is worth about 1% here, and is NOT worth a serialize.cs API addition on this evidence. Recorded as ranked-and-refused, not as a follow-on. TWO CONSEQUENCES OF FUSION, named in the file rather than found later: - A stream BOTH truncated inside a run AND carrying an out-of-range value before the truncation now surfaces the stream's overflow error where the per-field form surfaced the range refusal. Both refuse the packet; the set of ACCEPTED streams is unchanged, which is the property that matters at the trust boundary. Rust, Go, Java, Dart and JS-flat already carry it. - A failed run leaves the destination object untouched where the per-field form left the fields before the failure updated. Both are partial states a failed read gives no contract over (SPEC §5). NEGATIVE CONTROL, run rather than assumed: with flatMaskLit forced to emit no mask — invariant 2 deliberately broken — `make test` produces 65 failure lines across the backends. The gate that proves this form does see it. MEASURED, against the previous commit, same box, median of 3 sittings: bench_mixed write 3.30 -> 3.30 M msg/s (unchanged, as expected) bench_mixed round_trip 1.18 -> 1.54 M msg/s (+30%) and against main: bench_mixed write 2.10 -> 3.30 M msg/s (+57%) bench_mixed round_trip 0.97 -> 1.54 M msg/s (+59%) Wire bytes unmoved: full `make test` green across nine backends, the degenerate corpus included, test/cs and test/cs-ludicrous (which pin the corrupt-wire refusals) among them, and the bench's golden + per-variant round-trip gates pass at corpus_id 6b213fbfa1a03a99. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Every runtime line in the preamble already records `<sha> (<branch>)`; the schema commit line recorded a bare sha. Two failures that closes, both of which this repo has already had: a sweep that silently measured the wrong tree and a CSV that could not have said so, and a published result whose sha stopped resolving the moment the branch was rebased. The branch name survives the rebase. One line in run.sh (it reuses commit_of, which the runtime lines already use) and its law-home in BENCH-STANDARD.md §3.5, beside the provenance clause the same class of defect earned in August. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

…sittings Both halves committed, --only cs, taken back to back in the same window, twice. Preambles carry the schema commit AND its branch on the after half; the before half runs main's own run.sh, which predates that line, so the file says so in its own note. bench_mixed, cs, family gen, checks=always (M msg/s): path before max after max before median after median write 2,104,563 3,342,814 2,103,454 3,334,630 1.59x round_trip 992,155 1,594,245 990,865 1,586,844 1.60x Against the quiet-box reference sweep of the same day and the same corpus_id (bench/results/2026-09-01-arm64-macbook.csv, cpp round_trip median 3,467,551; cpp write median 7,287,687), on the round_trip statistic the published ledger reads at: cs before 350% of generated C++ (the published ledger row: 361%) cs after 219% and on write, 347% -> 219%. The `bits` family rows in this pair are NOT attributable to the change and the files say so where a reader will meet them: BitsBench.cs exercises serialize.cs's raw BitWriter/BitReader and no generated code at all, yet its rows still move 11% across the halves at sub-3% spread. That can only be process-level — JIT method placement and cache state after a differently shaped bench_mixed pass in the same process. An open question, not a result. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

staticcheck QF1002 on CI: a two-arm `switch { case w == 64: default: }` is an if. Generated output is byte-identical — the regenerated tree is unchanged and `make test` stays green across nine backends. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

gafferongames · 2026-09-01T00:30:17Z

Adversarial review complete: MERGE.

The reviewer rebuilt main's compiler and ran a ~930,000-trial reflection differential against this branch's emitter (shipped corpus + a probe schema covering the 384-bit cap edges, chunk straddles, negative-min offsets, union arm shapes): zero wire-byte, verdict, bits-consumed, or error differences. The 65-failure mask negative control reproduced to the exact digit. Regeneration deterministic, no wire goldens moved, all three CSV stamps resolve, headline ratios recomputed from the committed CSVs and confirmed (1.59x/1.60x, 219% on the same-day cpp reference).

Four advisory findings, none wire-affecting, filed as follow-on issues after merge:

The go: the flat word codec — 712% to 248% of generated C/C++ #198 guard (1:1 piece↔item) is a property of code shape with no gate — a C# twin of TestFlatFallbackReEmitsItemsNotPieces is owed BEFORE lever F makes classification 1:N.
The fusion-consequence comment understates its class (wrong const / nonzero reserved also qualify, reproduced 35x on the shipped corpus with identical verdicts) + a dropped SPEC §4.3 citation in the flat read path.
batchEntry's b == 0 basis is false two ways (TFixed/128-bit score as scalars but delegate; densityUnion takes the heaviest arm's b, not max) — pre-existing miscount, now load-bearing for a ruling.
The zero-alloc "assert" prints to stderr and fails nothing — gate it or rename it.

Also noted: the batch-entry retraction from BenchPacket/RegainAfterAlign/TestData is a real scope change the body does not disclose; covered by land-and-expand but recorded here.

…d refusal cites SPEC §4.3 — closes #213 (#230) From the #208 review (finding §4 + the §10 nit): the consequence bullet named only out-of-range values, but any refusal the run's own validation raises qualifies — a wrong const and nonzero reserved reproduce it on the shipped corpus with identical verdicts. And the flat read path's reserved comment dropped the SPEC citation its const twin and the per-field path both carry. One emitted line moves (the citation); goldens re-pinned; wire untouched. Co-authored-by: Rowan Claude <rowan@mas-bandwidth.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

…loses #215) (#231) From the #208 review (finding §10): the comment said 'must be 0' and nothing failed on nonzero. Nonzero now Fail()s the leg with both figures, so a future allocation goes red in the sweep instead of scrolling past on stderr. Proven both directions in the quick leg: clean run 0/0 green; a planted 16-byte escape-proof allocation FAILED the leg with the exact figure. Co-authored-by: Rowan Claude <rowan@mas-bandwidth.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

#236) Two holes from the #208 review, both now closed: TFixed and the 128-bit family score as delegated sites (every WriteBatch.SerializeFixed/ Int128/UInt128 overload Sync/Recaptures — they were counted as plain scalars), and densityUnion's bulk count is the max over arms taken separately (the b==0 entry gate asks whether a delegated site CAN run in the body; a light arm carrying a string answers yes even under a heavier scalar-dense sibling). Consequence: the entry-batch gate now refuses the delegating shapes it was admitting through the holes — RealWorld's RealPacket and the ludicrous fixed/128 types lose batched entries (16 batch cores retracted across write+read). The measured surface is untouched by construction: generated/bench/cs BenchMixed does not change, and a quick cs leg on this branch reproduces the sitting-2 rates (round_trip 1.603M vs 1.586-1.594M, within noise) with the wire gate and the zero-alloc gate green. Co-authored-by: Rowan Claude <rowan@mas-bandwidth.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

…3% of generated C++ (#237) * js flat: single-word 32-bit write merge replaces the two-lane staging pair Profile (node --prof, bench_mixed --quick): 51% of ticks inside writeBenchMixedFlatProduction itself, 27% in V8 runtime calls it makes. The per-field merge carried two data-dependent branches (the s<32 lane split and the 64-bit flush) plus an s>0 guard; the single 32-bit staging word does one shift-or, one literal add, and one flush branch storing one word. Micro (interleaved, node 26, M2): 1.44x on a mixed-width group of ten. On the leg (quick, production): write 1.35 -> 1.49 M msg/s (+10%), round_trip 0.77 -> 0.82 (+6.5%). Wire bytes unchanged: an LSB-first packer into consecutive little-endian 32-bit words emits the identical byte stream; golden + 64-variant re-encode gates and test/js (both modes, degenerate corpus included) green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * js flat: write-side BigInt lane discipline — the DataView scratch consumes, nothing allocates The profile's 27% V8-runtime bucket was BigInt: every BigInt '&', '>>', asUintN-that-truncates and Number() on the write path allocates a BigInt and calls the runtime, where a DataView setBigUint64 consumes the value into plain memory without allocating. Micro (node 26, M2): 128-bit split 15.0 -> 84.3 M ops/s via two scratch stores + one shift (5.6x); sub-32 truncation 116.9 -> 718.9 M ops/s via scratch wrap (6.1x); the existing <=64 scratch form measured already optimal (674 M ops/s) and keeps its shape minus the redundant asUintN(64) wrap (ToBigUint64 IS that wrap; for the >64 half, shift-then-wrap equals the asUintN(128) high half). Reworked: emitWriteWideOffset takes the raw BigInt offset and routes all four width classes through the scratch; flags <=32 wire mask through the scratch instead of Number(asUintN); the >32-storage int32-range truncation becomes SC.getInt32 (ToBigUint64 wrap + signed lane == asIntN(32) for every BigInt). wideOffsetWidth deleted — nothing reduces ahead of the store anymore. Checked-mode guards unchanged. Leg (quick, production): write 1.49 -> 2.22 M msg/s (+49%), round_trip 0.82 -> 1.01 (+23%). Golden + 64-variant gates and test/js{,-ludicrous} green in both modes. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * js flat: wide (>64-bit) read assembly through the scratch — two halves, one shift-or The read twin of the write-side lane discipline: the 96/128-bit assembly built its BigInt from four BigInt(v) constructions chained with three shift-ors (seven allocating steps); it now assembles each 64-bit half in the number domain through SC.setUint32 pairs and joins two getBigUint64 loads with a single shift-or. Micro: 12.5 -> 40.3 M ops/s on the 128-bit assembly (3.2x, node 26, M2). Leg (quick, production): round_trip 1.01 -> 1.06 M msg/s, derived read 1.86 -> 2.07; write untouched within spread. Gates green both modes. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * js flat: chunked write merges — adjacent constant-width fields pack into one staged word The family's grouping lever (go #198's runs, cs #208's word codec) in the form this backend can hold: adjacent <=32-bit fields whose value expressions are pure (field loads, hoisted element refs, literals — never a mutable temp) pack into one 'v' with literal RELATIVE shifts, then merge once. Relative offsets inside a chunk are static even where the absolute cursor is dynamic, so the lever reaches the loop bodies carrying most of the corpus wire (the 80-element stat array drops from two merges per element to one; the entity body from ~14 to ~6). Chunks never cross a scope boundary or a statement-form merge — wire order is emission order, enforced by chunkFlush at every loop, branch, switch, align, and scratch-based field; a temp-based piece (the >32-storage int32-range 'n') flushes around itself so no piece outlives its inputs. Leg (quick, production): write 2.22 -> 2.48 M msg/s (+12%), round_trip 1.06 -> 1.11 (+5%). Wire identical: golden + 64-variant gates, test/js{,-ludicrous} both modes, degenerate corpus — green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * js flat: string/bytes bodies merge four bytes per staged word Measured first as a hand-patched ceiling on the generated leg (+3.6% write), then landed in the emitter: the byte-copy loop for string(N)/ bytes(N) takes 4 bytes per 32-bit merge with a byte-at-a-time tail. The counter lives in a brace scope so two byte fields can share a nesting depth. Leg (quick, production): write 2.48 -> 2.55 M msg/s, round_trip 1.11 -> 1.12. Gates green both modes. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * js flat: re-pin the SOURCE goldens for the restructured emission Source goldens only — testdata/wire is untouched by this round (the stop-the-line invariant), and the full nine-language make test is green against the unchanged wire pins. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * bench/results: the js writes round's certification pair (family precedent: the pair rides with the round) before 1dba683 (main) / after 3b3e6f6 (the round), one sitting 39s apart, spreads 0.22-0.82%. The before leg reproduces the standing ledger row within 0.3% — the pair's validity check. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: Rowan Claude <rowan@mas-bandwidth.com>

…k-bytes path; 363% -> 221% of generated C++ (#238) * dart: chunked write merges — 64-bit lanes, single-merge wide fields Adjacent constant-width pure-expression fields pack into one staged word with literal relative shifts (the family lever, #198/#208/#237), capped at Dart's native 64-bit lane where the js number domain held it to 32. Wide 33..64-bit fields merge once instead of twice — their wire bits are contiguous, so the bytes are identical by the concatenation argument. Leg (quick, arm64 M2): write 1.894 -> 2.380 M msg/s (1.26x), round_trip 0.974 -> 1.088 (1.12x). Golden gate + 64-variant re-encode green; dart conformance suites green with asserts on. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * dart: bulk-bytes write path — the #165 lead String/bytes bodies and bare [N]uint8 arrays stop paying one merge per byte: static runs of 8 or fewer bytes queue as chunk pieces (joining neighboring fields in one merge), longer and dynamic runs take 4 bytes per merge with a byte tail. No alignment requirement: LSB-first merging of the concatenation equals merging the bytes in sequence at any bit position, so the fix reaches the unaligned loadout too. Micro (AOT, M2): 4-byte grouping 2.33x, per-byte baseline 1.00x. Leg (quick): write 2.380 -> 2.499 M msg/s (+5.0%), round_trip 1.088 -> 1.104 (+1.4%) — bulk is only 3.65% of bench_mixed's wire by construction; the lever is for the real packets the issue names. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * dart: chunked read windows — one load and one tail branch per run, not per field The reader kept reloading its 64-bit window at every field. A load now folds the byte-interior shift in and guarantees 57 valid bits (in the tail arm, every bounds-proven remaining bit), so consecutive constant-width fields extract at literal relative shifts from one window through a compile-time ledger. Wide 33..57-bit fields extract in one masked read — their groups are contiguous on the wire. Short bare byte arrays unroll into the same windows (#165's read twin). The ledger invalidates at every dynamic bitsRead move and scope boundary. Micro (AOT, M2): chunked window 1.35x over per-field reload. Leg (quick): round_trip 1.104 -> 1.367 M msg/s (+23.8%), write flat; derived read 2.01 -> 3.03 M msg/s. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * dart: grouped counted loops — k elements share one lane per iteration A counted array of static-width elements unrolls k = lane/elemBits elements per iteration (64-bit chunk lane on the write side, 57-bit window on the read side), with a remainder loop. The chunk accumulator and window ledger span the unrolled elements, so short elements merge together and extract from one load — bench_mixed's 80-element stat array drops from one merge and one window load per element to one per three. Hoisted element refs get unique names (e0, e0g1, e0g2) so the unroll shares one scope; correctness rides the same ledger/flush machinery as straight-line code. Bounded first by a hand-patched generated leg (+10% round_trip on the read side alone). Leg (quick): write 2.493 -> 2.856 M msg/s (+14.6%), round_trip 1.367 -> 1.592 (+16.5%). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * dart: re-pin source goldens for the chunked emission Wire goldens untouched — testdata/wire byte pins hold in every leg; the re-pin is the SOURCE form only (SPEC §3.1's deliberate-change path). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * dart: range-over-int in the emitter's unroll loops (modernize) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * bench/results: the dart round's certification pair before 1,900,191/972,818 (reproducing the standing ledger row within 0.6%), after 2,857,586/1,591,336; spreads 0.07-2.28%; one sitting, corpus 6b213fbfa1a03a99. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: Rowan Claude <rowan@mas-bandwidth.com>

gafferongames and others added 5 commits September 1, 2026 09:19

gafferongames changed the title ~~cs: the composition levers — union batch cores and scoped batches (write +34%, round_trip +11%)~~ cs: the codec pass — union batch cores, scoped batches, and the flat word codec (write 1.59x, round_trip 1.60x; 361% -> 219% of generated C++) Sep 1, 2026

gafferongames marked this pull request as ready for review September 1, 2026 00:03

gafferongames merged commit 9051689 into main Sep 1, 2026
8 checks passed

gafferongames deleted the cs-word-codec branch September 1, 2026 00:30

gafferongames mentioned this pull request Sep 1, 2026

cs: the batch density model counts delegation truthfully (#214) #236

Merged

gafferongames mentioned this pull request Sep 1, 2026

js: the write round — flat writes 1.89x, round_trip 1.47x; 461% -> 313% of generated C++ #237

Merged

gafferongames mentioned this pull request Sep 1, 2026

dart: the laggard round — chunked lanes, windowed reads, the #165 bulk-bytes path; 363% -> 221% of generated C++ #238

Merged

gafferongames mentioned this pull request Sep 1, 2026

The generated-code optimization round: forensics-corrected baselines, then per-emitter profiling passes #170

Open

Navigation Menu

Uh oh!

cs: the codec pass — union batch cores, scoped batches, and the flat word codec (write 1.59x, round_trip 1.60x; 361% -> 219% of generated C++) - #208

cs: the codec pass — union batch cores, scoped batches, and the flat word codec (write 1.59x, round_trip 1.60x; 361% -> 219% of generated C++)#208
gafferongames merged 6 commits into
mainfrom
cs-word-codec

gafferongames commented Aug 31, 2026 •

edited

Loading

Uh oh!

gafferongames commented Sep 1, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

gafferongames commented Aug 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

1. MEASURE FIRST — where the time actually was

2. THE LEVERS, ranked, and what each returned

3. Lever A' — the measured refusal, stated

4. Lever E — refused on evidence, not deferred

5. Discipline

Uh oh!

gafferongames commented Sep 1, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

gafferongames commented Aug 31, 2026 •

edited

Loading