Skip to content

One bench, one corpus: retire the §1.2 example-corpus rows from every runner - #204

Merged
gafferongames merged 5 commits into
mainfrom
retire-example-rows
Aug 31, 2026
Merged

One bench, one corpus: retire the §1.2 example-corpus rows from every runner#204
gafferongames merged 5 commits into
mainfrom
retire-example-rows

Conversation

@gafferongames

Copy link
Copy Markdown
Contributor

Retires the §1.2 example-corpus bench rows — rigidbody_moving,
rigidbody_at_rest, chat, test, inputpacket, shipcreate,
probe_header, probebits, probearray, testdata and real_packet
from the c, cpp, go, rust, cs and js runners, together with every
hand-written pin, vary function, field check and per-shape driver that fed
them. They were the last hand-coded shape measurement code in the harness.

The owner ruling reaches them directly: "there should be only a single
schema bench: Bench.schema, it generates per-language stuff that is how we
measure how efficient serialize and schema is per-language"
/ "We do all
profiling inside schema now."
#199 removed the hand-coded Bench-corpus
shapes and family rt; this removes the second corpus that rode the full
sweep beside them. BenchMixed is now the one measured shape in every one
of the nine runners.

real_packet rides along even though its shape lives in
bench/corpus/RealWorld.schema rather than examples/: it is a second
bench schema
with its own hand-written pin and vary body, which is exactly
what "a single schema bench: Bench.schema" forbids.

What is deleted, and what is kept

Deleted: measurement code only — 480 lines from bench/c/bench_main.c, 535
from bench/cpp/bench_main.cpp, and the equivalents in go, rust, cs and js;
the retired rows' lines in bench/inline-budget.txt (a budget line for a leg
that produces no verdict fails the gate as a vanished verdict); the
corpus list in bench/tools/relative.go; the now-dead example /
realworldcorpus dependencies in go.mod, Cargo.toml and the C/C++
include paths.

Kept, because make test proves they are load-bearing:
examples/*.schema, bench/corpus/RealWorld.schema, all of generated/
and every testdata/wire/*.bin. The cross-language conformance suite pins
these instances byte-for-byte — test/main.cpp, test/c/main.c and the go
/ rust / cs / js / java / dart / elixir port suites for the example
goldens; test/bench/main.cpp, test/bench/c_main.c and the java / js /
dart / elixir suites for real_packet. Nothing under examples/,
generated/ or testdata/ is touched by this PR. The two generated units
that no longer have a bench consumer keep their compile gates:
generated/bench/rust-realworld its own cargo build in the Makefile, and
generated/bench/cs/realworld a Compile item in schemabench.csproj
documented as a compile gate with no bench reader (issue #80's lesson —
that unit once shipped uncompilable while every other gate stayed green).

Family bits (§1.4) is out of scope and untouched.

Also, both ruled out of #199's report: §1.8's SUSPENDED bench_wstring
definition is deleted
. schema defers wstring (SPEC §4.10) and #188 owns
growing the language; a suspended row for a construct the language cannot
express is fossil, not a record. When schema expresses wstring the row is
defined then, against what the language actually emits. (#199's other ruled
item — bench/LOCK's control 2 naming rows that no longer exist — is a
separate PR touching only that file, per LOCK's own amendment protocol.)

Verification

  • make test green, 26 conformance legs, nine languages. Nothing
    deleted here was load-bearing for a test; the fixtures that were, stayed.
  • All nine runners build and run bench_mixed with their §1.5 golden
    gates green: c, cpp, go, rust, cs, js, java, dart, elixir.
  • corpus_id 6b213fbfa1a03a99 on all nine --quick legs, unchanged.
  • bench/run.sh --quick prints the two-column table: c 98%, cpp 100%,
    rust 153%, java 156%, cs 360%, dart 392%, js 469%, go 581%, elixir 3957%.
  • bench/tools/inline-gate.sh selftest passes all 25 fixtures;
    inline-gate.sh leg {c,cpp,go,rust,cs} all PASS, 0 unguarded.
  • Emitters and generated/ byte-unchanged: a full make from clean
    leaves the working tree empty. This PR's diff touches bench/ only.

Ledger — and yes, this is another #194 sub-era boundary

Receipts and the full write-up: bench/results/exrows-air/.

before → after → before for the c and cpp legs in one sitting, then an
A/B/A/B reproduction sitting per language (a macOS media-indexing daemon
held a core and put sitting 1's cpp control over §2.3's 15% spread
threshold; the failed-control CSVs are landed rather than dropped).

leg row before after control drift
c bench_mixed write 6.43 M/s 6.90 M/s +7.4% (control +0.1%)
c bench_mixed round_trip 3.48 3.43 -1.3% (control -0.9%)
c bitpacker write/read flat, ≤0.4%
cpp bench_mixed write 7.07 7.08 +0.2%
cpp bench_mixed round_trip 3.46 3.37 -2.8%
cpp bitpacker read 87.4 k/s 84.9 k/s -2.8%

bench_bitpacker is byte-identical in both trees in both languages
the diff moves only its call site's position in the file — and cpp's still
moves 2.8%. That is the negative control. C's +7% reproduces in two
independent windows against controls under 1%.

This commit is a #194 absolute-ledger SUB-ERA BOUNDARY for the c and cpp
legs.
Removing ~500 lines from each TU moves layout again, exactly as #199
found one merge earlier. Do not read C's +7% as an improvement or cpp's
-2.8% as a regression; do not smooth it and do not chase it.

The full c/cpp sweep's corpus_id moves a4bc52bae5343e60
6b213fbfa1a03a99 — required, since the leg no longer loads the eleven
retired goldens, so the full sweep's id now equals --quick's. §5.3 rule 2
mechanically refuses every ratio across the boundary, which is the loud
re-pricing §1.7 rule 3 demands. The goldens themselves are byte-unchanged.

Note for PR #200

This PR deletes exactly the hand-coded shape measurement code that #200's
shape gate exists to keep out, so that gate's ledger counts will change
under it. bench/tools/shapegate and SHAPE-GATE.allow are #200's and are
not touched here.

🤖 Generated with Claude Code

gafferongames and others added 5 commits August 31, 2026 22:27
The §1.2 rows — rigidbody_moving, rigidbody_at_rest, chat, test,
inputpacket, shipcreate, probe_header, probebits, probearray, testdata,
real_packet — were the last hand-written pin / vary / field-check / sink
code in either runner, over examples/*.schema rather than Bench.schema.
The owner ruling PR #199 carried ("there should be only a single schema
bench: Bench.schema") reaches them; #199 flagged them as a distinct
excision and this is it.

Gone from both runners: the pins, the vary mappings, the hand-written
per-shape driver (bench/c/bench_message.inc and the cpp bench_message
template), the call sites and the main() locals. bench/c/bench_message.inc
had no other caller and is deleted.

Two consequences the deletion surfaced:

  - the cpp runner's #189 workaround (#undef SCHEMA_UTF8_VALID_DEFINED /
    SCHEMA_INTERIOR_NULL_DEFINED before BenchWire.h) existed only because
    TWO schema units rode in one translation unit. Only the bench unit
    rides now, so the guards can never be pre-set and the workaround is
    dead. Removed with the includes that caused it.

  - --quick and the full sweep now differ only by family bits, so the cpp
    runner's early-return quick block collapses into the same straight
    line the c runner already had.

The examples/*.schema sources, their generated code and their wire
goldens are untouched: test/ pins them.

corpus_id for bench_mixed is unchanged at 6b213fbfa1a03a99, and it is now
the id of the full sweep too — the full leg loads no golden the quick leg
does not.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…js runners

The same excision as the c/cpp commit, in the four remaining runners that
carried it. Gone: the pins, the vary mappings, the hand-written per-shape
drivers (benchMessage / bench_message / BenchMessage, and in js BOTH tiers
— benchMessageFlat and benchMessage), the call sites, and the js
read-side sink folds (sinkOf*, boolBit, bigBit, sumBytes) whose whole
reason was hand-written per-shape observation.

The example and realworld generated units leave each runner's dependency
declaration with the code that named them: bench/go/go.mod drops the
example module and its replace, bench/rust/Cargo.toml drops the example
and realworldcorpus path deps, and bench/cs/schemabench.csproj drops
generated/cs/*.cs and generated/bench/cs/realworld/*.cs. Each runner now
compiles exactly the one generated unit it measures.

Two rust clippy allows went with the code that needed them
(field_reassign_with_default was the pin_* idiom, too_many_arguments was
bench_message's eight parameters); clippy reports no new warning without
them.

--quick and the full sweep now differ only by family bits in every runner,
so the four early-return quick blocks collapse the way c already read.

corpus_id 6b213fbfa1a03a99 for bench_mixed, verified in all six.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…dget ledger

run.sh: the c and cpp legs compile ONE generated unit now, so
-Igenerated/cpp, -Igenerated/c and -Itest leave both flag sets, and the c
leg's presence check moves from generated/c/TypesWire.h to
generated/bench/c/BenchWire.h — the header it actually needs.

inline-budget.txt: the 140 example-shape lines go. This is not cosmetic:
inline-gate FAILS on a budgeted leg that produces no verdict ("a vanished
verdict must not pass"), so leaving them would have red-lit CI, the same
way bench_mixed's lines did on #199. Only the 14 bitpacker lines remain.

relative.go: the `corpus` list (the eleven §1.2 names) is deleted; `rel`
medians over benchCorpus, and the relative table's second column is
round_trip rather than read — the gen family's two MEASURED paths under
§2.9. Read is derived to stderr and emits no row, so ratioing it would
have printed a table of dashes.

inline-verdict.sh: BENCH_MAP is now EMPTY, and the c and cpp gen
attribution goes with it. That machinery keyed off the hand-written
driver — cpp on the bench_message<T> instantiation, c on the
bench_message_<suffix> frame plus a bench_message.inc line range — and
bench_message.inc no longer exists, so the c branch would have hard-failed
on its missing marker greps. Both branches now compute the bits family's
noinline-loop verdicts and nothing else. UNIT_MAP and the per-language
symbol spellings stay: they are what the #177 follow-on needs to
repopulate the map, and inline-gate's unit fixtures still exercise them.
The five surviving BENCH_MAP consumers skip a blank line so an empty map
iterates zero times instead of once.

inline-gate.sh selftest: all fixtures pass.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
BENCH-STANDARD.md §1.2 is now a RETIREMENT RECORD, the shape §1.3 already
uses for the three stress shapes: it names the eleven rows, quotes the
ruling that reaches them, and says plainly what stays and why —
examples/*.schema, their generated code and their testdata/wire goldens
are pinned by the conformance suites in up to nine languages, so they are
fixtures, not bench rows. §1.1's family table drops examples/*.schema as a
corpus source.

§1.7 keeps its law and its dated audit, with one paragraph saying where
rule 1 is discharged today: BenchMixed is the corpus's realistic snapshot,
and real_packet — the row that used to discharge it — is a conformance
fixture now.

§1.8: the bench_wstring definition is DELETED, not left SUSPENDED. schema
defers wide strings (SPEC §4.10), so no schema type, no generated codec
and no golden can carry the row, and the family that would have —
hand-written rt — is retired. A suspended row for a construct the language
cannot express is fossil; growing the language is issue #188's, and the
row gets defined against what the language actually emits when it does.
The section is now "The string row".

§1.5's reference points at bench_datadriven by name rather than at line
numbers that have moved twice this week.

bench/README.md loses the eleven-row benchmark-set table and the porting
instructions that told a new runner to transcribe pin_* and vary_*; the
runner contract now says what it actually is — port bench_datadriven, name
no field of the shape, gate variant 0 against the golden and round-trip
every variant. The go, rust and cs READMEs follow.

ONE thing goes back: bench/cs/schemabench.csproj keeps compiling
generated/bench/cs/realworld even though no row reads it. `make test` runs
`cd bench/cs && dotnet build`, and C# has no unit-local build file, so this
project is the ONLY gate proving that generated unit compiles — issue #80
is the record of that unit shipping uncompilable while every other gate
stayed green. It is marked COMPILE GATE ONLY in the csproj and in the
README, with the right home named: a cs conformance leg pinning
real_packet, the one backend that has none while java, js, dart and elixir
all pin the golden.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The measurement receipts the retirement owed, plus the audit's findings on
the four commits before it.

Ledger (bench/results/exrows-air/): before -> after -> before for the c and
cpp legs in one sitting, then an A/B/A/B reproduction sitting per language
after a macOS media-indexing daemon put sitting 1's cpp control over §2.3's
15% spread threshold. C's bench_mixed write moves +7%, reproduced in two
independent windows against controls under 1%; cpp's byte-identical
bench_bitpacker read moves -2.8%. Neither is a serializer change: this is a
#194 absolute-ledger sub-era boundary, the third today.

Doc corrections:

* §1.2 named test/bench/main.cpp and the java/js/dart/elixir suites as
  real_packet's conformance pins and missed test/bench/c_main.c, which pins
  it too. The claim is load-bearing — it is the reason the golden stays.
* three paragraphs in bench/{go,rust,cs}/README.md were left unreflowed by
  the edits before this one.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@gafferongames
gafferongames merged commit 0514fc4 into main Aug 31, 2026
7 checks passed
gafferongames added a commit that referenced this pull request Aug 31, 2026
#199 deleted family rt and the §1.3 hand-coded shapes, #204 retired the §1.2
example-corpus rows from every runner, #201 deleted the decomp.cpp scratch
harness. The ledger's ratchet caught all of it: nine stale counts, seven
entries matching nothing.

Name hits under bench/ drop from 2080 to 146, and of the 29 remaining in the
nine runner legs NOT ONE is hand-coded shape measurement — every one is a
comment, a generated-symbol callsite, or an English word that is also a corpus
field name. The nine legs are shape-blind. The gate now guards a clean tree
instead of a shrinking debt.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
gafferongames added a commit that referenced this pull request Aug 31, 2026
…out (#200)

* The shape gate: a CI refusal for hand-coded shape measurement

Owner concern, 2026-08-31: "if you have any other profiling per-language
that is hand coded ... I don't trust this and it will drift. We have to
lock this in."

bench/tools/shapegate extracts the shape vocabulary from bench/corpus and
refuses four things: a corpus identifier named under bench/, a timing
primitive anywhere outside the sanctioned runner and tool directories, a
bench-shaped source path outside them, and a shape's wire size written
down as a literal.

bench/SHAPE-GATE.allow is the complete register of what does not yet
comply, with exact counts that ratchet: growth fails, and so does leaving
a count too high once the debt is paid.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* shapegate: satisfy errcheck and staticcheck ST1005

The rule text moves out of the error value and under it, which is where
a reader wants it anyway.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* shape gate: rebase onto main — the debt is paid, the register says so

#199 deleted family rt and the §1.3 hand-coded shapes, #204 retired the §1.2
example-corpus rows from every runner, #201 deleted the decomp.cpp scratch
harness. The ledger's ratchet caught all of it: nine stale counts, seven
entries matching nothing.

Name hits under bench/ drop from 2080 to 146, and of the 29 remaining in the
nine runner legs NOT ONE is hand-coded shape measurement — every one is a
comment, a generated-symbol callsite, or an English word that is also a corpus
field name. The nine legs are shape-blind. The gate now guards a clean tree
instead of a shrinking debt.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* shapegate: skip dist/, and state the two limits the rebase exposed

Two findings from re-proving the gate on a fully provisioned tree.

dist/ is the Makefile's pinned toolchain drop — the Dart SDK, the JDK, OTP,
Elixir. It is gitignored and absent on CI, which uses setup-dart/setup-java, so
the shape-gate job never met it. On a developer machine that followed the
Makefile's own dist/ instructions the gate refused with 42 findings, every one
of them inside a downloaded toolchain: the Dart SDK ships lib/core/stopwatch.dart
and Mix ships profile.fprof.ex. `make shape-gate` was unusable on exactly the
trees that can run the whole bench. dist/ joins the skipDirs list.

The package doc gains the limitation #198 demonstrated: this gate guards
MEASUREMENT, not CORRECTNESS. The Go emitter wrote a fixed scalar array twice —
32 wire bytes where the other eight languages write 16 — and Go-to-Go
round-trips passed clean because both ends shared the defect. A shape-blind
runner driving a defective emitter is still shape-blind and still passes here.
Issue #203 records the corpus blind spot. It also now says plainly that a shape
name in a comment counts as a hit, which is deliberate.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* ci: keep the cpp-lock comment attached to the cpp-lock job

The shape-gate job was appended after cpp-lock's explanatory comment and
before cpp-lock itself, orphaning the comment above the wrong job. Same two
jobs, comments back with their own.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
gafferongames added a commit that referenced this pull request Aug 31, 2026
The PR carried a before/after table and a 0.9977x negative control with NO
committed CSV, and #199 retired the shapes those numbers were taken over,
so they could not be re-run on main. These two files are that evidence,
re-taken on the shape that survives — bench_mixed, family gen — and
committed before AND after, which is #183's precedent.

One sitting, back to back, on the tree as it stands after #204 rewrote the
runners. Nothing changed between the halves but generated/bench/go: main's
Go emitter output for the before half, this branch's for the after.

  go        write 1.35 -> 2.67 M msg/s (1.98x), round_trip 0.60 -> 1.41 (2.36x)
  headline  583% -> 248% of generated C++ on run.sh's own statistic (2.35x)
  controls  geomean 0.9940x over 16 rows — every other leg, unchanged code,
            compiled identically in both halves (min 0.9426, max 1.0339)

corpus_id is 6b213fbfa1a03a99 in every row of both files: the runner hashes
the goldens it loads, and Degenerate.schema's golden is not one of them.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
gafferongames added a commit that referenced this pull request Aug 31, 2026
* go lever 1: fold the ranged-integer READ path to generation-time bit counts

The write side already folded its bounds; reads still called the runtime's
SerializeInt/SerializeInt64, which derive the bit count from min/max on EVERY
call (a BitsRequired per field) and carry the bounds as arguments. Both are
compile-time constants of the call site.

Folded, reads drop onto SerializeBits — whose wrapper the Go compiler inlines,
where the ranged entry points are 168/328 cost units against an 80-unit budget
and never inline. The headroom refusal moves into generated code with the same
error, ErrValueOutOfRange, and is elided where the range fills its bit width.

Wire bytes unmoved; corpus_id 6b213fbfa1a03a99 unchanged.
Measured, M2, bench_mixed, same sitting: write 1.37 -> 1.39 M msg/s (+1.5%),
round_trip 0.61 -> 0.62 (+2.6%).

The small size is the finding: the cost is the CALL COUNT, not the call's
arguments. That is what lever 2, the flat word codec, attacks.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* go lever 2: the self-contained flat word codec

Maximal runs of statically-sized pieces are gathered, field values computed
into locals and OR'd into word-sized chunk locals at literal shifts, and one
whole chunk handed to the stream per call. Reads fuse their bounds checks per
chunk and take ONE sticky-error test per run instead of one per field.

WriteMixedEntity goes from 14 stream calls to 3.

Two deliberate deviations from the Rust template (#183), both Go physics:

  1. CHUNKS ARE 64 BITS, not 32. SerializeBits64 places a whole word in one
     call with one bounds check (its two tryWriteBits are inlined into it),
     where two SerializeBits are two calls. Wire-identical: SerializeBits64
     splits low-dword-then-remainder, which IS the 32-bit chunk order.
     Chunks of 32 bits or fewer still go through SerializeBits, which inlines.

  2. EVERY PIECE IS MASKED TO ITS WIDTH. serialize.go's write path MASKS a
     too-wide value; the Rust runtime only debug_asserts, so #183 had to
     REFUSE. Masking is what keeps the Go form observably identical.

And one coverage win the Rust template could not have: COMPRESSED FLOATS DO
NOT BREAK A RUN. This emitter already folds quantization into generated
arithmetic ending in a plain bit write, so quantized fields are ordinary
pieces — where Rust's live in the runtime, which is why its compressed-float
shapes barely moved.

Wire bytes unmoved (testdata/wire clean, corpus_id 6b213fbfa1a03a99).
Only Go outputs regenerated; every other language byte-identical.
Green: schema_test, _random, _ludicrous, C and C-ludicrous, bench and bench_c
(including the 6 bench_mixed refusal vectors), test/go, test/go-ludicrous,
and the fuzz leg.

Measured, M2, bench_mixed, same sitting, on top of lever 1:
  write      1.39 -> 2.08 M msg/s  (1.50x)
  round_trip 0.62 -> 1.01 M msg/s  (1.63x)
Negative control: the hand-written rt rows, same source throughout, held at
1.22/1.20 against a 1.23/1.22 baseline.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* go lever 3: place nested struct fields inline instead of calling per element

With the per-field calls gone, the profile's next item was the CALL boundary
itself: bench_mixed carries up to 80 MixedStat and 8 MixedEntity elements, and
each paid a function call plus its own sticky-error load. WriteMixedStat was
12.4% of samples for 18 bits of work, and at cost 134 against Go's fixed
80-unit budget it will never inline itself.

An array whose element is a struct that flattens whole now has its element
body placed directly in the loop. The element's own Write/Read functions are
still emitted — the public surface is unchanged — they are simply no longer
the path the generated caller takes.

Measured, M2, bench_mixed, same sitting, on top of lever 2:
  write      2.08 -> 2.27 M msg/s  (+9%)
  round_trip 1.01 -> 1.11 M msg/s  (+10%)
Control rows held at 1.17-1.20 write / 1.18-1.21 read against the 1.22/1.20
baseline. Wire unmoved, conformance green.

Cumulative over the three levers: write 1.37 -> 2.27 (1.66x),
round_trip 0.61 -> 1.11 (1.82x).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* go levers 4 and 5: nested struct FIELDS and small fixed arrays join the run

Two coverage extensions on the same mechanism, both cheap once a run can be
built against an arbitrary base expression:

  4. A nested struct field that flattens whole contributes its fields to the
     PARENT's run, so they pack into the parent's chunks rather than paying a
     call and starting fresh chunks of their own.

  5. A small fixed array of scalars is a static run of bound*width bits like
     any other, so it unrolls into the enclosing run instead of looping one
     stream call per element. Capped at 128 bits so generated code stays
     proportional to the schema, and byte-aligned [N]uint8 keeps the bulk-copy
     path, which is faster than either form.

Measured, M2, bench_mixed, same sitting, on top of lever 3:
  write      2.27 -> 2.32 M msg/s  (+2%)
  round_trip 1.11 -> 1.16 M msg/s  (+4.5%)
Control rows 1.20/1.21 against the 1.22/1.20 baseline. Wire unmoved,
conformance green.

Cumulative over five levers: write 1.37 -> 2.32 (1.69x),
round_trip 0.61 -> 1.16 (1.90x), blended 1.83x.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* go: flatten only where it REDUCES the stream call count

The full per-shape leg caught two rows going the wrong way: rigidbody_moving
and rigidbody_at_rest both lost 9% on write. Mechanism, found in the emitted
code — RigidBody is all float64, so every piece is exactly one whole chunk.
The run packed into as many chunks as it had fields, removing no call, while
ADDING a materialized local per field and an address-taken chunk local, where
the per-field form had simply passed the struct field's own address.

The policy is now the honest one: flatten when ceil(bits/chunk) is fewer calls
than the run has bit-carrying pieces, and otherwise leave the per-field form
alone. That makes the transform non-regressive on call count by construction
rather than by luck.

bench_mixed is unaffected (write 2.35, round_trip 1.16 — its runs all reduce
calls); the rigidbody shapes return to the per-field form.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* go lever 6: pack consecutive array elements into shared chunks

The profile after the first five levers put the generated code's own packing
arithmetic at 15% of samples and the runtime's bit placement at 30% — the
per-field calls were gone, but bench_mixed still spent most of its calls on
eighty 18-bit stat elements, each carrying 18 of a possible 64 bits.

An array of a struct that flattens whole is now emitted as a K-at-a-time loop
over a run spanning K elements, followed by a remainder loop. K is chosen as
the element count that minimises chunks-per-element within the run cap and an
eight-element unroll cap; for the 18-bit stat element that is 7, and eighty
elements fall from 80 stream calls to 25.

Measured, M2, bench_mixed, same sitting, on top of lever 5:
  write      2.34 -> 2.66 M msg/s  (+14%)
  round_trip 1.15 -> 1.40 M msg/s  (+21%)

Cumulative over six levers, against current main in one sitting:
  write      1.38 -> 2.66 M msg/s  (1.93x)
  round_trip 0.61 -> 1.40 M msg/s  (2.30x)
  blended    1182 -> 545 ns/msg    (2.17x)

Named behaviour change, the same class the run form already carries: a group's
refusals all run before the group packs, so a refused write has emitted fewer
bits than before. Which values are refused, and with which error, is
unchanged. Wire bytes unmoved, conformance green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* go lever 7: raise the run cap to its measured peak, and scope the grouped loop

Two things, found together when the cap was moved.

THE BUG: the grouped array form declares its index outside the loop, so two
grouped arrays in one function collided on i. It only surfaced once the cap
let a second array group, but it was latent from lever 6. The grouped loop now
carries its own block, as the runs already do.

THE CAP: measured on the corpus shape rather than inherited from the Rust
template's 256. bench_mixed write / round_trip, M msg/s, one sitting:
  256  -> 2.66 / 1.40
  384  -> 2.78 / 1.49   <- peak
  512  -> 2.75 / 1.46
  1024 -> 2.62 / 1.43
The curve peaks and falls: past the peak a run holds more field values live
than the register file has room for and they spill, which is the cost the cap
exists to bound. 384 also lets the 135-bit entity element group two at a time.

Cumulative over seven levers, against current main in one sitting:
  write      1.38 -> 2.78 M msg/s  (2.01x)
  round_trip 0.61 -> 1.49 M msg/s  (2.44x)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* bench: the gocodec quick sweep — go at 255% of generated C/C++

Six legs measured in one sitting (java, dart and elixir report ABSENT: this
fresh clone has no pinned dist/ toolchains and the box has no JRE). The
locked c/cpp reference reproduces at 100/104%, which is the control on the
sitting.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* go: range over int in the group loop (modernize lint)

Emitter output unchanged — regenerated with no diff.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* go: re-derive the invariant the fallback rests on — groups, not pieces

PR #183's Rust template maps one item to exactly one piece, so its fallback
(re-emit each of a run's pieces) IS re-emitting each of its items. This
emitter's levers made classification 1:N — an unrolled fixed array
contributes one piece per element, a flattened nested struct contributes
pieces naming ITS fields at a different base expression — and the fallback
was inherited verbatim. Three defects followed from that one gap:

  - `type Pair { values [2]float64 }` emitted the element loop TWICE on both
    paths: 32 bytes where every other language writes 16. Go-to-Go round
    trips passed, so nothing went red.
  - a nested struct split by the run cap emitted `value.C` on the OUTER
    type — a compile error, or, under a name collision, the silent
    serialization of the wrong field.
  - a bare `type Vec2 { x float64; y float64 }` carried an unused "math"
    import: needsMath was set during SPECULATIVE classification and never
    cleared when the run fell back.

A run now accumulates whole ITEMS as flatGroups and splits only on item
boundaries, so the fallback re-emits items against the base each item owns.
flatPiece no longer carries an ir.Item at all — a piece is a bit-placement
recipe and nothing else, which makes re-emitting one structurally
impossible rather than merely avoided. needsMath moves into the float
pieces' emit/read closures, which run at emission.

Wire is unmoved and generated/ is zero-diff on the corpus: none of the three
shapes exists in it. That blind spot is issue #203, addressed next.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* corpus: the degenerate arrangements, C++-pinned in all nine languages (#203)

Three emitter defects — one of them silent wire corruption — reproduced on
one-line schemas, and the entire conformance corpus caught none of them.
It grew from real game shapes, and a real shape always carries enough
neighbouring fields to mask the arrangements a run/chunk emitter keys on.
The corpus's realism is precisely what made it blind.

examples/Degenerate.schema carries twelve shapes that are deliberately
unrealistic: a fixed scalar array as essentially the whole message
([2]float64, [2]uint64, [2]int64, [1]T, [N]T landing on a chunk boundary,
an array plus a trailing field, two arrays back to back); a bare two-float
unit with no other math consumer in the file; and a nested struct as the
ONLY field, as the FIRST field, and straddling the Go emitter's run cap.

They are pinned as one wire golden, testdata/wire/degenerate.bin, and every
one of the nine legs is held to it: the stream legs (C++, C, Go, Rust, C#,
JS) write the twelve into one stream, and the whole-message legs (Dart,
Java, Elixir) concatenate their twelve buffers. That equality is why every
type in the file is a whole number of bytes wide — the file's header says
so, because it is load bearing.

internal/codegen/golang states the same properties directly against
maxRunBits, so re-tuning that constant cannot quietly retire the coverage.
Verified red first: on the pre-fix emitter all four assertions fail, and
generated/go/Degenerate.go does not compile ("math" imported and not used;
value.C undefined on TrioStraddle).

The new file caught a SECOND emitter defect on sight, the same class as the
Go one: the Dart backend declared all four conversion-scratch views whenever
any was needed, so a file converting only float64 carried an unreferenced
_u32 and `dart analyze` refused it. The views are now emitted per
conversion.

Existing wire goldens are unmoved; the corpus protocol id moves, as adding
a file must, and is re-pinned.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* bench/go: state the --cpuprofile flag in the usage line and refuse instead of panicking

The Go leg is not untouched: the flat word codec's conviction — ~86% of
generated-codec time inside the runtime's per-field bit calls — came from
this flag. It stays, because the next round will want it and a profiler
kept in a scratch fork of the leg is a profiler that rots. It is an
iteration instrument: it changes nothing about what is measured, and no
timed row is taken under it.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* test/c: dirty the degenerate read targets before reading them

gcc's -Wmaybe-uninitialized (CI, -Werror) cannot see that a read function
stores every field, and clang does not raise it, so the new block built
clean here and refused there. The rest of the leg already dirties its read
targets — a read that skips a field must be caught — so this block does the
same, with the same 0xEF fill.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* bench: the paired same-sitting go A/B, with its controls, committed

The PR carried a before/after table and a 0.9977x negative control with NO
committed CSV, and #199 retired the shapes those numbers were taken over,
so they could not be re-run on main. These two files are that evidence,
re-taken on the shape that survives — bench_mixed, family gen — and
committed before AND after, which is #183's precedent.

One sitting, back to back, on the tree as it stands after #204 rewrote the
runners. Nothing changed between the halves but generated/bench/go: main's
Go emitter output for the before half, this branch's for the after.

  go        write 1.35 -> 2.67 M msg/s (1.98x), round_trip 0.60 -> 1.41 (2.36x)
  headline  583% -> 248% of generated C++ on run.sh's own statistic (2.35x)
  controls  geomean 0.9940x over 16 rows — every other leg, unchanged code,
            compiled identically in both halves (min 0.9426, max 1.0339)

corpus_id is 6b213fbfa1a03a99 in every row of both files: the runner hashes
the goldens it loads, and Degenerate.schema's golden is not one of them.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* go: bind the run once in flush, and say why an oversized group takes the per-field form

Comment and shape only — generated output is byte-identical. flush() built
the run twice, once to ask worthFlattening and once to emit it; the
oversized-group branch had no note saying no classifier produces one today.
The orphaned flatFieldPiece doc above flatArrayElemPiece now sits on the
function it describes.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
gafferongames added a commit that referenced this pull request Aug 31, 2026
The published figures were unverifiable: `git log origin/main..HEAD --
bench/results/` was empty, so no re-runner could check them. #198, #199
and #204 all committed their before/after data this week; this matches
that.

Both halves are a fresh paired A/B taken in one sitting on the M2 Air
after the rebase, three minutes apart, corpus_id 6b213fbfa1a03a99 on
every row. The BEFORE half stamps main at 52691a0 and the AFTER half
elixir-swizzle at fef86ed — the last CODE commit on the branch, so both
stamps resolve on origin and reconstruct the tree that was measured.

Elixir, bench_mixed, family gen:
  write      135,696 -> 378,044 msg/s  (2.79x; spread 2.08% -> 0.22%)
  round_trip  95,413 -> 239,723 msg/s  (2.51x; spread 0.46% -> 0.38%)

As a share of generated C++ (§2.9, max rates): 3743% -> 1457%. Both
elixir rows are far inside §2.3's 15% noise gate, so neither is a row
bench/tools/relative.go would exclude.
gafferongames added a commit that referenced this pull request Aug 31, 2026
…#202)

* elixir #174: the profile, before any lever is pulled

Wall-clock decomposition of generated Elixir on the canonical shape
(bench_mixed, 438 bytes: 8 entities x 135 bits, 80 stats x 18 bits,
4 loadout bytes), M2 Air, OTP 29.0.5 / Elixir 1.20.4, 100k ops per
measurement, the same instances the bench rotates:

  write        7.344 us/op
  read         3.561 us/op   (round_trip 10.748)

  write splits:
    stats loop, 80 elements     2.632 us    32.9 ns/stat  (36%)
    entities, 8 elements        2.970 us   371.5 ns/ent   (40%)
    header + footer + loadout   1.742 us                  (24%)

The write path is 2x the read path, and 325 binary appends per message
is what it is made of: the emitter's mergeW flushes every whole byte
after EVERY field, so one bs_append BIF call rides each of the 131
static field sites plus 8x14 + 80x2 + 4x1 loop sites.

eprof agrees on the ordering (stats 38.7%, entities 23.7% of write) and
is not quoted for magnitudes: 162000 traced calls into the stats loop
inflate exactly the function the wall clock says is hot.

Nothing measured here lands in serialize.elixir. The generated Elixir
codec has no runtime dependency at all, so every lever below is a
schema-level emitter change and #170's routing question does not arise.

* elixir: lever A — one binary append per group, not per field

The emitter knew every field width statically and threw the knowledge
away: mergeW flushed the scratch's whole bytes after EVERY field, so a
438-byte bench_mixed message cost 325 bs_append BIF calls.

mergeW now carries a GROUP. It merges into the scratch and flushes only
when the next field would pass the budget, and flushW closes the group at
every barrier that observes data or scratch_bits — the write function's
tail, an align, the bytes of a string, a loop helper's call and its own
element tail, and the joins of a branch or a union case. flushW is a
no-op when no group is open, which is what makes a barrier free where one
is already closed.

The budget is 52 bits and the number is not a taste. The BEAM's fixnum is
60-bit signed, so an intermediate at or above 2^59 costs a heap bignum; a
flush leaves at most 7 bits behind, so 7 + 52 = 59 is the whole envelope.
Measured on the read side of the same shape, a 72-bit window is SLOWER
than the shipped 40-bit one (29.4 vs 27.6 ns/element) — boxing costs more
than the flush it would save, so the group stops at the boundary.

MEASURED, wall clock, canonical shape, 100k ops (before -> after):

  write            7.344 -> 5.208 us/op     1.41x
    stats loop      32.9 -> 22.3 ns/stat
    entities       371.5 -> 223.2 ns/entity
  round_trip      10.748 -> 8.886 us/op
  read             3.561 -> 3.561 us/op     (untouched, as expected)

  bench --quick: write 0.13 -> 0.18 M msg/s, round_trip 0.09 -> 0.12

Static append sites in generated/bench/elixir: 131 -> 65; per entity
14 -> 3, per stat 2 -> 1; ~325 -> ~128 appends per message.

Wire bytes unmoved: corpus_id 6b213fbfa1a03a99, the full variant
round-trip gate green, test/elixir and test/elixir-ludicrous OK,
mix format --check-formatted clean.

* elixir: lever C — one window decode per group, not per field

rd/3 opened a fresh match context for EVERY field: 14 per entity, 2 per
stat. The generator knows every width statically, so readR now reads a
GROUP into rv once and cuts each field out with a static shift and mask.
The fused static run's own length sizes the group, so a short run keeps
the cheap 40-bit window and a long one takes rdw's 56-bit window.

Two windows, and the second one's width is the same fixnum argument the
write group's budget rests on. A 56-bit window less the 7-bit worst-case
offset is a 49-bit group and stays under 2^59; a 64-bit window would box.
Measured on this shape a 72-bit window is SLOWER than the shipped 40-bit
one it would replace — 29.4 vs 27.6 ns/element — so 49 is the ceiling and
not an arbitrary stopping point.

Reading a group wider than the fields it feeds is safe by construction:
rd and rdw never raise (the tail falls back to the bytes that exist), and
bits past the bounds-checked run are discarded, never observed. Every
bounds check, range check, constant check and refusal is where it was.

rdBreak closes the group at every barrier — the read surface, a loop
helper's entry and every call to one, an align, the bytes of a string,
and the arms of a branch or a union case.

MEASURED, wall clock, canonical shape, 100k ops (after lever A -> after C):

  read             3.561 -> 2.384 us/op     1.49x
    stats           27.6 ->  ~15  ns/stat   (rd calls 2/elem -> 1)
    entities                               (rdw calls 14/elem -> 4)
  round_trip       8.886 -> 8.126 us/op
  write            5.208 -> 5.208 us/op     (untouched, as expected)

  bench --quick: round_trip 0.12 -> 0.13 M msg/s

Wire bytes unmoved: corpus_id 6b213fbfa1a03a99, the full variant
round-trip gate green, test/elixir and test/elixir-ludicrous OK,
mix format --check-formatted clean.

* elixir: lever B — one map read per scope, not one per field

Elixir's `.` on a struct is a get_map_elements of its own with its own
raise branch. The writer spent one per field: 14 for an entity, 2 for a
stat, ~30 at the top level. A scope now reads its fields ONCE —
`%{stat_id: e_stat_id, delta: e_delta} = e` — and every reference to
those fields is a local from there on.

Scope, precisely: a scope binds the fields it reads UNCONDITIONALLY at
its own level — its field items and the conditions of its branches. A
branch arm's fields are NOT bound at the enclosing level; the arm is its
own scope and binds them when it is taken, so a value the wire never asks
for is still never demanded of the caller. Below two fields the pattern
would not pay for itself and none is emitted.

Refusals are unchanged. A struct always carries every key, so the bound
locals are exactly as unconditional as the dotted accesses they replace,
and every range, count, mask and length check is where it was. The one
delta, named: a MAP missing a key now raises MatchError where it raised
KeyError — a raise either way, on exactly the same inputs, never a wrong
answer. The raise TEXT is unchanged: g.dsp resolves a local back to the
dotted access it stands for, so the message still reads "e.delta is above
the wire maximum" and never names a local the caller never wrote.

MEASURED, wall clock, canonical shape, 100k ops (after lever C -> after B):

  write            5.208 -> 3.202 us/op     1.63x
    stats loop      23.4 ->  15.4 ns/stat
    entities       212.6 ->  96.4 ns/entity
  round_trip       8.126 -> 5.466 us/op
  read             2.384 -> 2.276 us/op     (the read side binds locals already)

  bench --quick: write 0.18 -> 0.31 M msg/s, round_trip 0.13 -> 0.18

Cumulative over A + C + B, against the round's baseline:

  write            7.344 -> 3.202 us/op     2.29x
  read             3.561 -> 2.276 us/op     1.56x
  round_trip      10.748 -> 5.466 us/op     1.97x

Wire bytes unmoved: corpus_id 6b213fbfa1a03a99, the full variant
round-trip gate green, test/elixir and test/elixir-ludicrous OK,
mix format --check-formatted clean.

* elixir: lever E — the float32 step, and a declaration constant folded

Three changes to the compressed-float helpers, all of them removing work
that was provably redundant:

fr/1 lets the REFUSAL be the test. A float segment does not match a
non-finite pattern, so the finite path is one construction and one match
and never touches the exponent field; the second clause reads the sign of
exactly the patterns the first refused. NaN still maps by sign, as it did.

cf_quantize takes miv32 — the float32 of the step count — as a
generation-time literal instead of computing fr(miv * 1.0) on every call.
It is a declaration constant; folding it removes one of the six float32
steps. cf_decode takes it directly in place of the integer count, which
was only ever used to compute the same rounding.

trunc(Float.floor(x)) becomes floor(x): one BIF returning an integer
instead of a float floor and a truncation. The argument is finite by
construction (normalized is clamped to [0, 1] and miv32 is finite), so
the paths differ only in the class of an unreachable raise.

MEASURED, wall clock, canonical shape, 100k ops, and reported honestly:

  write            3.202 -> 3.18 us/op   AT OR BELOW the noise floor
  read             2.276 -> 2.19 us/op   ~3%
  round_trip       5.466 -> 5.30 us/op

The isolated micro said more (cf_quantize 203.6 -> 135.2 ns/call), and
the micro was wrong to say it: it passed fr as a closure, so every fr
call it removed was an inflated indirect call rather than the direct
local call the generated module makes. The in-situ number is the ruling.
The lever stays because it costs nothing and removes real work; it is
recorded as small, not as what the micro promised.

Wire bytes unmoved: corpus_id 6b213fbfa1a03a99, the full variant
round-trip gate green, test/elixir and test/elixir-ludicrous OK,
mix format --check-formatted clean.

* elixir: re-pin the four source goldens the emitter moved

Deliberate emitter change, and only the SOURCE goldens move: no file under
testdata/wire is touched by this commit or by any of the four levers, and
the full `make test` chain — every language's conformance suite, the wire
goldens, the fuzzers, the format refuser — is green.

* elixir: lever D — the static offsets the generator already knew

The group model carried its widths statically and then computed with them
at runtime anyway: every merge shifted by the scratch_bits VARIABLE and
added to it, and every flush divided that variable by eight to size a
binary segment whose width was therefore dynamic.

A write function starts at a known empty scratch and stays known until the
message's own data decides a length. Through that whole region the emitter
now tracks the offset itself, so:

  - a merge is one statement, `scratch = scratch ||| v <<< 16`, with a
    literal shift and no scratch_bits arithmetic at all; the first merge of
    an empty group is a bare bind, `scratch = v`
  - a flush is `data = <<data::binary, scratch::little-size(4)-unit(8)>>`
    with a LITERAL segment width — the form the BEAM's binary construction
    is built for — plus a literal shift, in place of four statements
  - an align that lands on a boundary emits nothing, and one that does not
    appends the residual byte unconditionally instead of testing for it
  - the function's tail is `data` or `<<data::binary, scratch>>` outright,
    never `if scratch_bits != 0`
  - scratch_bits is not even bound where the surface never needs it

Staticness is a property the emitter EARNS and gives up honestly. It is
given up where a loop helper is called (how many elements rode is the
message's business) and where a branch's arms end on offsets that disagree;
in that case each arm publishes the offset it reached and the emitter goes
back to maintaining the variable. It is REGAINED at every align, which
lands the position on a byte whatever the data did. Where every arm of a
branch or a union case does agree, the offset stays static past the join
and scratch_bits does not ride the join's tuple at all.

Behaviour is unchanged: the same statements in the same order over the same
values, with the arithmetic the generator can do moved to generation time.
Every range, count, mask and length check is where it was, and every raise
carries the text it carried.

Wire bytes unmoved: no file under testdata/wire is touched, corpus_id
6b213fbfa1a03a99, the full variant round-trip gate green, test/elixir and
test/elixir-ludicrous OK, mix format --check-formatted clean, and the whole
nine-backend `make test` chain green. Only the four SOURCE goldens move,
re-pinned here with the emitter that moved them.

MEASURED: in the sitting's A/B, recorded on the PR with the round's table.

* elixir: lever J — a scalar array element sized its own read window

Lever C sizes each window decode by the fused static run it sits in, and
the run fuser is reached through emitReadItems. A scalar array element is
not: readHelper hands it straight to the scalar read, so the run was zero —
unknown — and readR fell back to its widest window. A one-byte element was
opening the 56-bit window to take eight bits.

That is not merely a wider mask. rdw needs seven bytes ahead of the
position to match its window and rd needs five, so the wide window drops
into the :binary.decode_unsigned tail fallback two bytes sooner — and a
scalar array is very often the LAST thing in a message, which is exactly
where that boundary lies.

A scalar element's own width is the run, so readHelper sets it. The loadout
loop now reads `rd(data, bits_read, 8)` where it read `rdw(data, bits_read,
49)`. Nothing else moves: a struct or union element still goes through
emitReadItems and fuses its own runs, and readR's choice of window was
already free to be any width that covers the field.

Wire bytes unmoved: no file under testdata/wire is touched, corpus_id
6b213fbfa1a03a99, the full variant round-trip gate green, test/elixir and
test/elixir-ludicrous OK, mix format --check-formatted clean, the whole
nine-backend `make test` chain green, and the two source goldens the
emitter moved re-pinned here.

MEASURED: in the sitting's A/B, recorded on the PR with the round's table.

* elixir: lever K — one append for a whole clause of elements, not one each

The write loop appended once per ELEMENT: 80 bs_append calls for the stats
array of one bench_mixed message, 4 for the loadout. Lever A closed the
group across the fields of an element; it could not close it across the
call boundary between elements, because a helper is entered at an offset
the caller's element count decides.

Measured in isolation, that boundary is where the write path's cost is.
Twelve appends of one 32-bit segment each cost 103.6 ns; the same 48 bytes
in three appends of four segments cost 79.0, and in one append of twelve,
73.3. The same twelve appends with a DYNAMIC segment width cost 103.7 —
identical. The append is the expense; the arithmetic around it is not.

So a clause takes SEVERAL elements off the list. k is chosen by the group
budget the fixnum boundary already fixed — the most whole elements whose
widths fit in 52 bits, capped at four so one array field cannot cost
unbounded generated code — and the k element bodies merge into one group
and flush once. The single-element clause behind the wide one is the
remainder, so a list length never has to divide anything, and an element
whose width the wire decides keeps one clause per element as before.

  stats  (18 bits): 2 elements per clause, 80 appends -> 40
  loadout (8 bits): 4 elements per clause,  4 appends ->  1

Nothing about an element's emission changes. The clause names its slots e1
and e2 where it used to have just e, and the raise TEXT is held: the
display map resolves every slot back to "e", so a message still reads
"e.delta is above the wire maximum" and never names a slot the caller never
wrote. Every range, count, mask and length check is where it was.

MEASURED, wall clock, the canonical shape from the committed variant
corpus, 64 rotating instances, 150k ops, 4 runs x 4 interleaved passes,
best per pass, median of passes:

  write            3.097 -> 2.65 us/op      1.17x
  round_trip       5.268 -> 4.74 us/op      1.11x
  read             2.132 -> 2.09 us/op      (untouched, as expected)

Wire bytes unmoved: no file under testdata/wire is touched, corpus_id
6b213fbfa1a03a99, the full variant round-trip gate green, test/elixir and
test/elixir-ludicrous OK, every generated module compiles without a
warning, mix format --check-formatted clean, the nine-backend `make test`
chain green, and the source goldens the emitter moved re-pinned here.

* elixir: lever L — one window decode for a whole clause of elements

The read loop's other half of lever K. Lever C reads a GROUP into one
window and cuts each field out of it with a static shift and mask, but a
loop helper is a function boundary, so a group could never span two
elements: 80 window decodes for the stats array, 4 for the loadout.

A read clause now decodes k elements under ONE window. k is chosen by the
wide window's usable width — the most whole elements whose widths fit in
49 bits, which is the same fixnum boundary the write budget rests on,
capped at four — and the clause carries a guard on the remaining count,
with the single-element clause behind it as the remainder.

Two things had to give way for a window to span elements. The element
emitters now take the variable they bind, so a clause can hold e1 and e2
without either shadowing the other. And the run fuser no longer overwrites
an OUTER run: a named element's own scope would otherwise size the window
to one element and undo the clause. Where no outer run is open the fuser
behaves exactly as it did.

  stats  (18 bits): 2 elements per rdw window of 36 bits, 80 decodes -> 40
  loadout (8 bits): 4 elements per rd window of 32 bits,   4 decodes ->  1

Reading a window wider than one element is safe on exactly the grounds
lever C established: rd and rdw never raise, the tail falls back to the
bytes that exist, and bits past the bounds-checked run are discarded and
never observed. The bounds check itself is unchanged — the call site
proved count * elem for a counted array, and an unbounded run still checks
its own span before it reads it.

MEASURED, wall clock, the canonical shape from the committed variant
corpus, 64 rotating instances, 150k ops, 4 runs x 4 interleaved passes,
best per pass, median of passes, against lever K:

  read             2.089 -> 1.838 us/op     1.14x
  round_trip       4.736 -> 4.409 us/op     1.07x
  write            2.703 -> 2.691 us/op     (untouched, as expected)

Wire bytes unmoved: no file under testdata/wire is touched, corpus_id
6b213fbfa1a03a99, the full variant round-trip gate green, test/elixir and
test/elixir-ludicrous OK, every generated module compiles without a
warning, mix format --check-formatted clean, the nine-backend `make test`
chain green, and the source goldens the emitter moved re-pinned here.

* elixir: the emitter answers modernize and gofmt

Four findings, all of them this branch's own, none of them a behaviour
change: the two scope-binding map copies lever B introduced become
maps.Copy, and the two unroll caps levers K and L introduced become min.
The generator emits identical bytes for every schema in the tree.

CI's lint job runs modernize at @latest, so a check that ships tomorrow
lands on the next branch the same way; this is the branch paying for its
own four.

* elixir: re-pin Degenerate's source golden after the rebase onto main

#198 landed `examples/Degenerate.schema` and pinned its Elixir source
golden with the pre-round emitter, so this branch merging as-is turned
main red: `TestGoldenSourceElixir` refused
`testdata/golden/elixir/Degenerate.ex`.

The re-pin is a SOURCE re-pin only. `make update-goldens` rewrote every
wire golden under `SCHEMA_UPDATE_WIRE_GOLDENS=1` and every one came back
byte-identical, `testdata/wire/` has no diff, and the nine-backend chain
byte-compares `degenerate.bin` against the C++ pin in all nine legs and
is green. The new text is the round's own shapes: one map destructure
per scope (B) and literal-width flushes with `scratch_bits` gone (D).

* corpus: Clauses.schema and Joins.schema — the arrangements Degenerate cannot reach

Degenerate.schema's standing property is that every type in it is a whole
number of bytes. That is load bearing for what it catches and it is also a
ceiling: no clause boundary inside it ever lands mid-byte, so an emitter
that groups array elements picks the same group size on the write and the
read side of every type in the file. The Elixir round's write clause has a
52-bit budget and its read clause a 49-bit window, so on Degenerate the two
always agree and the disagreement is untested.

Clauses.schema picks element widths where they do not. At 13 bits the write
clause takes four elements (52, the whole budget) and the read clause three
(39, inside the window); at 17 it is three against two; at 26, two against
one. Counts run 0, below a clause, exactly a clause, one past it, and the
bound, so the remainder path is entered from every phase. It also carries a
fixed mid-byte array, grouping across a nested struct boundary, a union of
empty arms behind a tag, and string/bytes at zero, partial and full length
behind a 5-bit lead so the align inside them is a real barrier.

Joins.schema does the same to the static-offset state machine: arms that
agree and disagree on width, a branch with no else, a branch inside a
branch, an align that regains staticness on one path only, an array that
gives it up on one path only, unions of unequal arms at mid-byte offsets,
and a long static run after an align.

Unlike Degenerate these shapes are NOT byte-aligned, so one shared stream
would not equal a concatenation of the shapes written alone — and the Elixir
emitter returns each message as its own binary from bit zero, so it cannot
write a shared stream at all. Every shape is therefore written to its own
stream and flushed, and the golden is those concatenated. Every leg can
reproduce that, and each shape's bytes stay individually attributable.

C++ pins, Go and Elixir byte-compare; the remaining six legs follow. Adding
units to the corpus moves the example unit's protocol id, as #198 did when
it added Degenerate. No existing wire golden moved.

* corpus: the Rust, C and JS legs byte-compare Clauses and Joins

* corpus: the C#, Dart and Java legs byte-compare Clauses and Joins

All nine legs now hold the two units to the C++ pin. The corpus README
gains a row for each, stating what each reaches that Degenerate cannot.

* bench: commit both halves of the sweep the round's numbers come from

The published figures were unverifiable: `git log origin/main..HEAD --
bench/results/` was empty, so no re-runner could check them. #198, #199
and #204 all committed their before/after data this week; this matches
that.

Both halves are a fresh paired A/B taken in one sitting on the M2 Air
after the rebase, three minutes apart, corpus_id 6b213fbfa1a03a99 on
every row. The BEFORE half stamps main at 52691a0 and the AFTER half
elixir-swizzle at fef86ed — the last CODE commit on the branch, so both
stamps resolve on origin and reconstruct the tree that was measured.

Elixir, bench_mixed, family gen:
  write      135,696 -> 378,044 msg/s  (2.79x; spread 2.08% -> 0.22%)
  round_trip  95,413 -> 239,723 msg/s  (2.51x; spread 0.46% -> 0.38%)

As a share of generated C++ (§2.9, max rates): 3743% -> 1457%. Both
elixir rows are far inside §2.3's 15% noise gate, so neither is a row
bench/tools/relative.go would exclude.
gafferongames added a commit that referenced this pull request Sep 1, 2026
…220)

The emitted string helpers (schema_utf8_valid, schema_interior_null) are
namespace-local, but their include guards were TU-wide — so the SECOND
generated unit included into one translation unit lost its helpers and its
string wire functions did not compile. The guards now carry the package name
(SCHEMA_<PKG>_UTF8_VALID_DEFINED / SCHEMA_<PKG>_INTERIOR_NULL_DEFINED),
following the emitFlagAppendHelper pattern already in the emitter: one copy
per package per TU.

C is NOT defective and stays untouched: its helpers are file-scope static
with trailing-underscore names, so the TU-wide guard is exactly right there
(the second unit's functions call the first emission's definition).

Regression test: test/guard — two packages, both with string(N) fields,
generated at build time into build/ and included into ONE translation unit
(build/schema_test_guard). Red on the old emitter, green here.

The bench/cpp leg-local workaround (#undef of the two guards) was already
removed by #204 when the example units left the bench TU; nothing remains.

Licensed by bench/LOCK's one-shot carve (owner ruling 2026-08-31, #189).
Generated diff: guard lines only, cpp outputs only; wire bytes unchanged.

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants