The shape gate: a CI refusal that keeps hand-coded shape measurement out

[gafferongames]

Owner concern, 2026-08-31, verbatim: "My concern is if you have any other profiling per-language that is hand coded. I don't trust this and it will drift. We have to lock this in."

This is the lock. An estate-wide audit ran alongside it; the findings are at the bottom.

Rebased 2026-08-31 onto main@0514fc4. Everything below — every count, every pasted refusal — is from the rebased head, not the tree this PR was written on. What changed is the headline: the debt is paid.

THE DEBT IS PAID — the gate now guards a clean tree

When this gate was written on main@6e1002b its register carried 2080 name hits across twelve runner files, because every one of the nine legs hand-wrote the corpus shapes: a family rt transcription against the runtime API, plus a family gen pinned initializer and LCG vary mapping per shape per language. Nine hand-written approximations of one benchmark. That was the drift the owner named.

Three PRs landed since:

• One bench, one shape: retire family rt and every hand-coded shape in the Bench-corpus leg (#196) #199 deleted family rt and the §1.3 hand-coded shapes from every leg
• One bench, one corpus: retire the §1.2 example-corpus rows from every runner #204 retired the §1.2 example-corpus rows from every runner
• Delete decomp.cpp: the #170 scratch harness, not a maintained instrument #201 deleted bench/results/harness-contract-air/decomp.cpp, the scratch harness this PR had flagged for the owner's ruling

The ratchet caught all of it, in the direction that matters. The rebased gate refused with nine stale counts and seven entries matching nothing — debt paid without the register saying so, which is a failure by design. The register is now rewritten to the true numbers.

Across the nine runner legs the count falls from 1946 to 29, and — the part worth saying loudly —

not one of those 29 is hand-coded shape measurement.

Every remaining hit in the nine legs is one of exactly three kinds, and every ledger line says which:

kind	what it is	example
(comment)	the shape's name in a prose comment. The gate does not strip comments, deliberately — a hand-coded bench announces itself in its own header	// Java codecs over bench/corpus/Bench.schema's BenchMixed.
(callsite)	the generated type named where the shape-blind driver instantiates it and hands it to generated Write/Read. It writes no field and pins no value	bench_datadriven<bench::BenchMixed>( "bench_mixed", ... )
(prose)	an English word that is also a corpus field name — a known false-positive class named in the package doc's stoplist note	extra in "one extra untimed pass"

The nine legs are shape-blind. The estate has reached the owner's ruling — there is only a single schema bench: Bench.schema — and this gate's job changes from shrinking a debt to holding a clean tree. Any leg that grows one hand-written field, one pin, or one wire size fails on the next push.

What is left on the register is the corpus tooling (one deliberate copy, by design), the §1.5 golden pinner's bench-shaped path, and two numeric false positives. Nineteen entries, all named, all reasoned.

The rule, made mechanical

The estate has exactly one sanctioned benchmark: this repo's data-driven bench. Shape knowledge belongs in bench/corpus/*.schema and in the code the compiler generates from it. Hand-written runners are fine and are the design. Hand-written measurement of a schema shape is not, anywhere.

bench/tools/shapegate refuses four things:

check	what it refuses
names	a corpus identifier — type, enum, union, flags or field, in any case form — appearing under bench/
timing	a timing primitive anywhere in the repo outside the sanctioned runner and tool directories
paths	a bench-shaped source path outside those directories
consts	a shape's distinctive wire size or bit count written down as a literal under bench/

The vocabulary is extracted from the corpus, not written into the checker, so renaming a field starts guarding the new name in the same commit. The wire constants come from the committed goldens' own file sizes, so no comment format or emitter behaviour has to hold for them to be right. If either extraction comes back empty the gate refuses to run rather than passing blind.

bench/SHAPE-GATE.allow is the complete register of everything that does not comply, each line with an exact count and a reason. The count is a ratchet in both directions: more hits fails because the debt grew; fewer hits also fails, printing the number to write, because the debt was paid and the register must say so; and an entry matching no file fails as rot. That is not a claim — it is what happened on this rebase, and it is what produced the numbers above.

make shape-gate locally. CI job shape-gate, on push and pull_request — unlike cpp-lock this is a standing property of the tree, not a review-window freeze.

Negative control, re-proven on the rebased head

A gate proven on an older tree is proven on an older tree. All four plants were re-run against main@0514fc4 + this branch, each reverted to green after. Pasted verbatim.

Controls A and B are the originals. Controls C and D needed new subjects — the originals planted into bench/go/rt.go and bench/elixir/runner.exs's hand-written shape block, and #199 deleted both. Their replacements exercise the same two ledger directions on files that exist today, and are noted where they differ.

A. A plausible hand-coded bench in internal/perf/mixed_bench.c — clock_gettime, a 438-byte buffer:

SHAPE GATE REFUSAL

TIMING  internal/perf/mixed_bench.c
    2 hit(s), no ledger entry
    clock_gettime

PATHS  internal/perf/mixed_bench.c
    1 hit(s), no ledger entry
    a source file whose path names a benchmark, outside the sanctioned runner directories
exit status 1

B. Shape code sneaked into a sanctioned runner (bench/dart/main.dart, ledger cap now 3, not 39 — the cap fell with the debt, so the plant is caught by a much tighter fence than before):

NAMES  bench/dart/main.dart
    11 hit(s), ledger allows 3 — the exemption GREW
    BenchMixed (corpus: BenchMixed)
    ack_bits (corpus: ack_bits)
    ack_sequence (corpus: ack_sequence)
    aim_x (corpus: aim_x)
    aim_y (corpus: aim_y)
    aim_z (corpus: aim_z)
    client_id (corpus: client_id)
    crc_hint (corpus: crc_hint)
    ... and 1 more

CONSTS  bench/dart/main.dart
    2 hit(s), no ledger entry
    3504 (bench_mixed wire bits)
    438 (bench_mixed wire bytes)
exit status 1

C. Ledger rot — a ledgered file removed without touching the register. Changed subject: the original deleted bench/go/rt.go to simulate #199; #199 has landed and that file is gone, so the plant now deletes bench/elixir/main.exs.

LEDGER  bench/SHAPE-GATE.allow:60
    `names bench/elixir/main.exs 1` matches nothing — the file is gone or clean.
    Delete this line.
exit status 1

D. Debt paid, register not lowered. Changed subject: the original removed four hand-written field names from bench/elixir/runner.exs; there are no longer four hand-written field names in any leg to remove — which is the point of this whole rebase. The equivalent plant rewrites the one comment in bench/java/Main.java that names the shape, dropping its count from 6 to 5:

LEDGER  bench/SHAPE-GATE.allow:63
    `names bench/java/Main.java 6` is stale — the file now has 5.
    Lower the count to 5. The ledger only shrinks.
exit status 1

The refuse-to-run-blind paths, also tested — the gate must refuse rather than pass silently when an extraction comes back empty. Three runs against synthetic roots:

# corpus directory with no .schema files
no .schema files under bench/corpus/ — the gate refuses to run with an empty vocabulary

# a corpus that parses but yields nothing guardable
corpus parsed but yielded no guardable identifiers — the gate refuses to run blind

# no goldens and no declared bit total >= 32
no wire constants derived from testdata/wire/bench_*.bin or bench/corpus/ — the gate refuses to run blind

All three exit 1. On the real tree both extractions are non-empty: 161 corpus identifiers, 731 spellings, and the wire constants 49 110 112 135 156 160 392 438 3504 read from the four committed goldens (bench_ints 14 B, bench_bits 20 B, bench_packet 49 B, bench_mixed 438 B) plus the schema's declared bit totals. The 14- and 20-byte shapes fall below the minConst floor and are guarded by name only, exactly as the package doc says.

Green on the rebased head as committed:

shape gate
  corpus vocabulary : 161 identifiers, 731 spellings guarded
  wire constants    : 49 110 112 135 156 160 392 438 3504
  ledger            : 19 exemption(s)

clean — 19 file(s) carry shape knowledge, every one on the ledger

make test passes locally on the rebased head with the full pinned toolchain. gofmt, go vet ./... and go mod tidy -diff clean.

One defect the re-proof found: dist/

Worth recording, because it was invisible on CI and would have hit every developer.

dist/ is the Makefile's pinned toolchain drop — the Dart SDK, the JDK, OTP, Elixir — gitignored, and absent on CI, which uses setup-dart/setup-java instead. So the shape-gate job never met it. On a machine that had followed the Makefile's own dist/ instructions, make shape-gate refused with 42 findings, every one of them inside a downloaded toolchain: the Dart SDK ships lib/core/stopwatch.dart, Mix ships profile.fprof.ex, OTP ships clock_gettime.

The gate was unusable on exactly the trees that can run the whole bench, and green CI could never have told anyone. dist/ joins skipDirs alongside generated, vendor and node_modules, with a comment saying why. Found only because the re-proof ran on a fresh clone provisioned the way the Makefile documents.

What this gate CANNOT see

Stated plainly, because a gate that oversells itself is worse than none. Also in the package doc, where the next reader will find it.

• It guards MEASUREMENT, not CORRECTNESS. A shape-blind runner driving a defective emitter is still shape-blind and still passes here. go: the flat word codec — 712% to 248% of generated C/C++ #198 is the worked example: the Go emitter wrote a fixed scalar array twice — 32 wire bytes where the other eight languages write 16 — and Go-to-Go round-trips passed clean because both ends shared the defect. Silent wire corruption, and nothing in this gate can see it. Cross-language wire agreement is the conformance suite's job; issue The bench corpus cannot see fixed scalar arrays, bare float units, or run-cap-split nested structs #203 records the corpus blind spot that let it through. This gate makes the nine legs measure the same shape; it does not make the nine emitters agree on the bytes.
• It runs in THIS repository's CI. A hand-coded serialize benchmark in another repository is entirely outside its reach. See the recommendation below.
• Markdown is not scanned. A benchmark pasted into a fenced code block passes.
• Short and ordinary names are not guarded. The vocabulary drops identifiers under 4 characters and a stoplist of ordinary words, because x, if, delta, chat and health are all corpus field names and guarding them would fire on English prose. A bench written using only those names would pass — it would also be unreadable.
• Small wire sizes are not guarded. Constants below 32 are not distinctive, so the two smallest corpus shapes are guarded by name only.
• Comments count as hits, and that is deliberate. Several ledger entries are comments and say so. Teaching the gate to skip comments would teach it to skip the header a hand-coded bench announces itself in.
• It is lexical. Code that computes a field name or a size at run time to evade it passes, and only a human reading the diff would catch that.
• Gitignored trees are skipped — dist/, generated/, vendor/, node_modules/, build/, target/. Nothing there is committed, so nothing there can drift into the repo, but the gate is blind inside them by construction.

Covering the other repos

Nine sibling repos are deleting their in-repo benches (serialize #105, serialize.c #57, serialize.cs #36, serialize.dart #3, serialize.elixir #2, serialize.go #54, serialize.java #2, serialize.js #9, serialize.rs #66). Nothing stops a tenth from appearing.

The cheap cover, not built here: lift the timing and paths checks — the two that need no corpus — into a reusable workflow in mas-bandwidth/.github, and have each family repo call it. Those two checks are the ones that generalise; names and consts need a corpus and belong only here. That is a separate PR in a separate repo and should be ruled on separately.

The register today

Nineteen entries. Ten runner lines (comment / callsite / prose, as classified above), five corpus-tooling lines, two numeric false positives, two paths lines for the §1.5 golden pinner.

The register names, deliberately and permanently until superseded: bench/tools/variantgen and realpacket-gen (one copy of shape code each, which is the design — one copy replacing nine; #195's emitter supersedes variantgen), bench/tools/inline-gate.sh (shape names as symbol-map labels and disassembly fixtures — no serialization, no clock), and test/bench/ (the §1.5 golden pinner: correctness code with no clock, which is exactly the line this gate draws).

Two numeric entries are honest false positives, kept visible rather than special-cased: bench/java/Main.java carries three references to GitHub issue #156, which collides with a corpus shape's declared 156-bit total, and bench/c/bench_main.c states the 438-byte golden size in a comment. Special-casing either would put a hole in the numeric check to buy tidiness.

Resolved since the original PR: bench/results/harness-contract-air/decomp.cpp — the 294-line hand-coded harness this PR flagged for the owner's ruling, held on the register under all four checks so it could not be forgotten. #201 deleted it and kept the .md, which was the recommendation. Its four ledger lines went stale on this rebase and the gate demanded their removal by name.

Files touched

bench/tools/shapegate/main.go, bench/SHAPE-GATE.allow, .github/workflows/ci.yml (one job), Makefile (one target appended), bench/README.md (one section appended). Nothing else.

🤖 Generated with Claude Code