BitWriter: single-word 32-bit staging — family bits 1.18x, writeBytes 1.37x - #11
Merged
Conversation
|
All contributors have signed the CAA. Thank you. |
gafferongames
force-pushed
the
bitwriter-single-word
branch
from
September 1, 2026 15:30
1d48155 to
0a96806
Compare
…ratch Every merge carried a data-dependent lane split (s < 32), a nested s > 0 guard inside it, and a 64-bit flush branch storing two words. One 32-bit staging word with scratchBits in [0,31] does the same wire in one shift-or, one add and one branch storing one word. Byte-equivalence: an LSB-first packer into consecutive little-endian 32-bit words emits the identical byte stream — the old pair of setUint32(base) and setUint32(base+4) IS two consecutive 32-bit words. The invariant is that the staging word's bits at and above scratchBits are zero; `value << s` contributes bits [s, min(s+bits, 32)), a JavaScript shift drops the rest, and the flush recovers the dropped high bits as the next word's low bits. spill === 0 is the single case where the recovery shift would be 32 — a no-op in JavaScript, not zero — so it is taken as a literal zero. Scope: the four fused copies (the file's KEEP THE FOUR COPIES IDENTICAL rule holds, applied by one scripted edit that refuses unless it finds exactly four), the flush and reset invariants, and writeBytes, whose empty-scratch word boundary is now every 32 bits rather than every 64 — head and tail through the scratch drop from 7 bytes to 3. flushBits keeps the memory behaviour: it stores the pending word and, when that lands on an odd 4-byte word, zeroes the next, so the bytes past the written data are still only ever written as zeros over the same 8-byte span and the buffer-length contract is unchanged. Paired instrument, arm rotation, max of 7 rounds, three invocations, the BENCH-STANDARD §1.4 family-bits workload: 1.1832 / 1.2177 / 1.1825 against a +/-1.1% A/A null. The writeBytes path 1.3694 / 1.3801 / 1.3871 against +/-0.2%. Wire gate every run: 65,518 bytes byte-identical, read-back and cross read-back exact. node --test 285/0 fail, node production-tests.mjs 124/0 fail, both including the golden-wire battery. schema's test/js and its bench leg run green against this checkout through the SERIALIZE_JS override. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…artial word The pairing store sat inside `if (this.#scratchBits !== 0)`. That guard is false exactly when the last write ended on a 32-bit boundary — the merge had already stored the word and left scratchBits at 0 — so for bitsWritten ≡ 32 (mod 64) the flush did nothing and the second half of the 8-byte span kept whatever the caller's buffer held before. main always wrote the full span. Payload bytes were never affected; the exposure is a reused dirty buffer whose consumer reads past bytesWritten() (4 stale bytes on a transmitted 8-byte aligned buffer), and the class doc's "bytes past the end of the written data are only ever written as zeros" being false. The store moves outside the guard. After the guard the cursor is at word ceil(bitsWritten / 32) on both paths — the partial word was stored, or there was none to store — so an odd index means the span's second half has never been written either way. The store does not advance the cursor, so a second flushBits is a no-op rather than a walk off the end. Both class copies moved. Reproduced first, against origin/main as the reference: a tail oracle writes k bits into a 0xff-prefilled buffer for every k in [0,384] and compares the whole buffer against main. Unfixed, 6 divergent counts — k = 32, 96, 160, 224, 288, 352 — every one past bytesWritten() and never payload. Fixed, 0 of 385. test/writer-tail-span.test.js pins k=32 and k=96 by name, sweeps every k in [0,384], covers the writeBytes bulk-copy cursor and flush idempotence, and asserts the full shape (packet, zeros to 8*ceil(k/64), untouched beyond) so over-zeroing fails as loudly as under-zeroing. It joins production-tests.mjs under that file's membership rule: it asserts no dev-only caller validation, and both class copies carry the flush. Negative control: with the store back inside the guard it reports 9 failures and exits 1 in both modes. The lever's own numbers move with it and the ROUND-LOG now carries these: family bits 1.1743 / 1.1809 / 1.1816 (was 1.1832 / 1.2177 / 1.1825), writeBytes 1.3594 / 1.3605 (was 1.3694 / 1.3801 / 1.3871) — one unconditional branch per packet is the difference. node --test 290/0 fail, production 129/0 fail. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
gafferongames
force-pushed
the
bitwriter-single-word
branch
from
September 1, 2026 16:10
1682464 to
c103647
Compare
gafferongames
marked this pull request as ready for review
September 1, 2026 16:12
Contributor
Author
|
Adversarially reviewed with schema#244: the byte-equivalence algebra verified, 65,536-byte wire identity at zero differing bytes, all four staging copies sabotage-covered by exactly their own suites, and the review's one condition — the odd-word tail zeroing at bitsWritten ≡ 32 (mod 64) — fixed with the previously-missing test class (bytes past bytesWritten on a dirty buffer, k-sweep 0/385 divergent, negative control 9 red). The lever's numbers were honestly republished downward post-fix (bits 1.17-1.18x, writeBytes 1.36x — far over the ±1.1%/±0.2% nulls). Verdict: MERGE. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #10. The runtime half of schema's js READS round; its twin is
mas-bandwidth/schema#244, and the two PRs pair (see "Pairing").
BitWriterstaged into a two-lane 64-bit scratch (#scratchLo/#scratchHi,#scratchBitsin [0,63]) and flushed a pair of little-endian 32-bit wordsevery 64 bits. Every merge carried a data-dependent lane split (
s < 32), anested
s > 0guard inside it, and the 64-bit flush branch. One 32-bitstaging word with
#scratchBitsin [0,31] writes the same wire in oneshift-or, one add and one branch storing one word — the form schema's
generated flat tier retired the two-lane staging for in schema#237.
Headline (BENCH-STANDARD.md §1.4 family
bits: the 16-width table,227 bits/group, over a 65536-byte buffer — the estate's one bitpacker
workload; max of 7 rounds, three invocations):
bitswriteBytes(Before the review fix below these read 1.1832 / 1.2177 / 1.1825 and 1.3694 /
1.3801 / 1.3871; the flush's now-unconditional pairing store costs one branch
per packet. The figures above are the ones that describe the branch.)
The instrument — A/A NULL before any number
A paired instrument loads two RUNTIME checkouts as arms, each arm's timing
loop built from its own source text so the two never share a
SharedFunctionInfo or a feedback vector. One path per invocation, arm order
rotated by round parity, max of 7 measured rounds after two discarded warmups
in both orders. Every pair run gates before it times anything: both arms
write the same 65,518 bytes byte-identically, each reads back exactly what it
wrote, and each arm's READER re-reads the other arm's buffer.
The A/A null — both arms
origin/main, the second a copy on a different path— is 0.9965 / 0.9973 / 1.0113 on
bitsand 0.9978 / 0.9979 onbytes.Byte-equivalence, stated
An LSB-first packer into consecutive little-endian 32-bit words emits the
identical byte stream either way: the old pair of
setUint32(base)andsetUint32(base+4)IS two consecutive 32-bit words. The invariant is that thestaging word's bits at and above
scratchBitsare zero;value << scontributes bits [s, min(s+bits, 32)) — a JavaScript shift drops the rest —
and the flush recovers the dropped high bits as the next word's low bits.
spill === 0is the single case where the recovery shift would be 32, a no-opin JavaScript rather than zero, so it is taken as a literal zero. With
sin[0,31] no shift here is ever the mod-32 no-op the two-lane form needed guards
for.
Scope, exactly as the issue named it
CheckedBitWriter.writeBits/.tryWriteBits,ProductionBitWriter.writeBits/.tryWriteBits. The file's own KEEP THEFOUR COPIES IDENTICAL rule holds: the edit was applied by one script that
refuses unless it finds exactly four.
which is where the flush's memory guarantee was actually made true.
writeBytes, whose "the scratch is empty at a word boundary" argumentnow holds every 32 bits rather than every 64: head and tail through the
scratch drop from ≤7 bytes to ≤3, and the bulk-copy unit is 4 bytes. That is
where its 1.37x comes from.
The review condition (#11): the flush now pairs its span for EVERY bit count
The pairing store sat inside
if (this.#scratchBits !== 0). That guard isfalse exactly when the last write ended on a 32-bit boundary — the merge had
already stored the word and left
scratchBitsat 0 — so forbitsWritten ≡ 32 (mod 64)the flush did nothing at all and the second halfof the 8-byte span kept whatever the caller's buffer held before.
mainalways wrote the full span.
Payload bytes were never affected. The exposure is a reused dirty buffer whose
consumer reads past
bytesWritten()— 4 stale bytes on a transmitted8-byte-aligned buffer — and, just as much, the class doc's "bytes past the end
of the written data are only ever written as zeros" being false.
The fix: the store moves OUTSIDE the guard. After the guard the cursor is
at word
ceil(bitsWritten / 32)on both paths — the partial word was stored,or there was none to store — so an odd index means the span's second half has
never been written, either way. The store does not advance the cursor, so a
second
flushBitsis a no-op rather than a walk off the end. Both classcopies moved.
Reproduced before it was fixed, with
origin/mainas the reference: atail oracle writes k bits into a 0xff-prefilled buffer for every k in [0, 384]
and compares the WHOLE buffer against the same write through main. Unfixed:
6 divergent counts — k = 32, 96, 160, 224, 288, 352 — every one past
bytesWritten(), never payload. Fixed: 0 divergent across all 385.The missing test class is now in the repo.
test/writer-tail-span.test.jspins k = 32 and k = 96 by name, sweeps every kin [0, 384], covers the
writeBytesbulk-copy cursor, and pins flushidempotence. It asserts the full shape — packet bytes, zeros to
8*ceil(k/64), untouched 0xff beyond — so over-zeroing fails it as loudly asunder-zeroing. It joins
production-tests.mjsunder that file's statedmembership rule (it asserts no dev-only caller validation, and both class
copies carry the flush). Negative control: with the store back inside the
guard the file reports 9 failures and exits 1 in BOTH modes.
Class and method documentation moved with the code — present state only, no
archaeology about the shape that was there before.
Gates
node --test— 290 tests, 0 fail (281 pass, 9 skipped), including thegolden-wire battery and the new tail-span suite.
node production-tests.mjs— 129 tests, 0 fail.STANDARD.mduntouched, so the vendored-copy job is unaffected.Pairing
schema's
test/jsandtest/js-ludicrous(both NODE_ENV modes) and its benchleg all run green against this checkout through schema's §3.5
SERIALIZE_JSoverride, at
corpus_id 6b213fbfa1a03a99. The two halves do not overlap andschema#244's certification pair shows it: the generated flat tier imports
nothing, so this lever adds nothing to the
bench_mixedround_triprow(1.2126x emitter-only, 1.2123x paired), and schema's emitter lands nothing on
family
bits.Honest note on that certification leg: its before and emitter-only after
bitpackerwrite rows printed at 47.7% and 49.3% spread — over BENCH-STANDARD§2.3's 40% INVALID threshold — so no ratio is taken from them there. This PR's
numbers stand on the paired instrument above, whose spreads are 0.13–1.13%;
the paired certification leg's own bitpacker write row (7,024 at 0.08% spread)
is consistent with them.
Landing follow-on, named not done: schema CI pins
SERIALIZE_JS_TAG: v1.1.0. Bumping that pin after this lands is a separate change in schema.🤖 Generated with Claude Code