Skip to content

Datatype nodes: an arm that re-forms its node keeps it whole, the counted take and wrap inline, a recursive drop, and flat defs lent a box - #2207

Draft
nicolas-abril wants to merge 5 commits into
bendlang:mainfrom
nicolas-abril:opt/shared-node-reuse-split
Draft

nicolas-abril wants to merge 5 commits into
bendlang:mainfrom
nicolas-abril:opt/shared-node-reuse-split

Conversation

@nicolas-abril

@nicolas-abril nicolas-abril commented Nov 18, 2026 •

Copy link
Copy Markdown
Collaborator

comp.ts is 67,588 ttok, 1,588 over the 66,000 cap (main 65,995; the five commits cost 638, 78, 442, 393 and 42), so repo-gate fails until that is settled (locally bun gates/repo.ts is 55 / 56, that one line). Measured on one Mac (M4), one thread, GPU off; other jobs held the machine at a load of 4 to 13 for the instruction counts and 2 to 4 for the cycle sweeps. The test and perf gates have not been run on the cluster.

Four changes to what a program does to a datatype node, one commit each, each with its own tests, and a fifth that fixes a use-after-free in the first (its own section below). They come from the Perceus benchmark set (ports on nicolas-abril/bend, branch bench/perceus): rbtree-ck (a red-black tree where every fifth tree is kept, so the type is shared), deriv, nqueens, and rbtree as the control whose type is never shared.

Instructions, G, after each commit:

main 1. arms 2. + inline 3. + drop 4. + lend
rbtree-ck 74.38 38.41 33.20 30.94 30.84 -59%
rbtree-ck-unsafe 49.25 33.03 27.91 25.65 25.55 -48%
deriv 12.32 12.30 12.09 10.89 10.89 -12%
nqueens 11.14 11.14 10.94 8.36 6.89 -38%
rbtree (control) 21.86 14.65 14.65 14.65 14.65 -33%
rbtree-unsafe (control) 11.21 10.39 10.39 10.39 10.39 -7%
cfold 0.766 0.766 0.766 0.746 0.746 -3%

Cycles, G, main against this PR, least to most over 16 placements of the hot code (a run of nops ahead of the segments, 4 bytes a step), at a load of 2 to 4:

main this PR per placement
rbtree-ck 14.40 to 14.56 6.46 to 6.58 2.20x to 2.25x
rbtree-ck-unsafe 10.37 to 10.44 5.80 to 5.84 1.78x to 1.79x
deriv 2.49 to 2.54 2.25 to 2.32 1.07x to 1.13x
nqueens 3.91 to 3.93 3.16 to 3.19 1.23x to 1.24x
rbtree 8.26 to 8.40 2.53 to 2.71 3.07x to 3.27x
rbtree, fresh cells carved 9 words apart 4.33 to 4.40 2.54 to 2.72 1.60x to 1.72x
rbtree-unsafe 2.73 to 2.78 2.22 to 2.26 1.21x to 1.25x

Checked rbtree on main loses nearly a factor of two to cache aliasing between 64-byte cells carved 8 words apart (the odd-stride row is the control: nothing else changes). With this PR it no longer does, so of its 3.1x, 1.6x to 1.7x is the change and the rest is that effect going away.

Peak memory: rbtree-ck 3,677 MB to 2,033 MB, rbtree-ck-unsafe 2,581 MB to 2,033 MB; the others unchanged.

1. An arm that re-forms its node keeps the node whole

def is_red(t: Tree) -> Tree & Bool:
  match t:
    case Node{Red{}, l, k, v, r}:
      (Node{Red{}, l, k, v, r}, True{})
    case t0:
      (t0, False{})

On main the first arm takes the node apart and builds it again, and so does the second (its default replays the pattern). A unique node is rebuilt in its own cell; a shared one is copied: five loads, a count added to each child, the old node dropped, a new node and later a new count cell, and the copy no longer shares with the tree it came from. Nested patterns do the same to every node they look into: balance1 opens both grandchildren to test their colour and rebuilds the ones that are not red, and checked rbtree's ins opens both children of every node on the path to read one key.

Now, in emit_match, a constructor arm of a boxed scrutinee is rewritten first (arm_self, the rewrite Nat and word matches already use): a K{f0, .., fn} that holds exactly the fields the arm bound, in their places, is the scrutinee. If after that every path of the arm uses the scrutinee, and no path uses a field that holds a box for anything else, the arm does not take the node: its word fields are read through the reference and the node is handed on as it came. An arm that opens with a match on a first field that is an enumeration (the colour) is split by that field, read through the reference, so that each colour is decided on its own: is_red becomes two reads, and balance1 takes a grandchild only when it is red.

Sound by construction: the rewrite replaces a node by an equal node, and the scrutinee is bound as an ordinary affine value (used twice, it is kept, like any other). Nothing is read after the scrutinee is handed on except word fields, which are locals by then. The rule needs every path to use the scrutinee because a path that only dropped it would trade main's inline free for a generic drop (kmeans showed it: +21% instructions before the rule had that clause).

Counts on rbtree-ck (main to this step): shared nodes copied 49.1M to 25.2M (the number of nodes on the insert paths, which is what a hand-written version copies), node allocations 246.5M to 126.8M, count cells 331.7M to 134.4M, count increments 93.7M to 49.1M. Checked rbtree, whose type is never shared: node allocations 245.7M to 126.0M, frame words 913M to 514M.

2. The take and the wrap of a counted node are inline on the host

span_fade (the take of a counted node) and rfc_wrap (a new count cell) are INLINE on the host and unchanged on the devices. The take is #2114's change (by Lorenzobattistela), included here rather than depended on, because by itself it costs lexer +2.9% instructions on this machine. lexer's 834M takes are of a constructor that has static nodes but is never shared; ctr_take is emitted for those too, and the inlined counted path, never run, makes each take two instructions longer. So node_fields now reads a constructor that is static but not hot in place, as it reads any other uncounted node (lexer -0.6%). That relies on the invariant the uncounted path already relies on: only a hot constructor's node is ever counted. ctr_take keeps its signature (the effect sources call it).

3. The host frees constructor nodes by recursion

term_sink goes to term_fall on the host: a constructor node is freed at once, its spine field (the one CID_T already names) loops, the other fields recurse, and a count is dropped inline for each child, so a child that is still shared costs no call. Past 64 levels, and for anything that is not a constructor node of at most 247 fields, it is term_drop's walk, unchanged; the devices only have the walk. Same decrements, same frees, in another order.

hashmap -20% instructions (its final drop; its counts do not change), deriv -10%, nqueens -24% (59.8M drops of a list that is still shared), rbtree-ck -7%.

4. A flat def is lent a box its caller still uses

def push(ok: Bool, queen: U32, xs: List<&2, U32>, xss: ..) -> ..:
  match ok:
    case True{}:
      (queen <> xs) <> xss
    case False{}:
      xss

append_safe uses xs again after the call, so on main it adds a count before every push and push drops it in the arm that does not cons: 59.8M atomic adds and 59.8M drops in nqueens where 4.67M lists are consed. Now a call to a flat def that does not loop, with an owned box the caller uses afterwards in a parameter the def owns, calls a second copy of the def (spin_push_2) in which that parameter is lent: where the def owns it, it adds the count (term_keep); where it only drops it, nothing happens; where it matches it, it reads through the reference. The caller seals the value first (rfc_seal: a count cell if it has none), so the def's term_keep never has to hand a new reference back.

Never more count operations than main: one add where main added one, none where main added one and dropped it. The caller holds its own reference across the call, which runs on the caller's thread.

nqueens: count increments 59.8M to 4.67M, decrements 64.5M to 9.35M.

Bend's suite and the koru kernels

bench/runtime, 17 programs, instructions in millions (one thread, GPU off, least of 3 interleaved runs). The worst is bfs, +0.24%.

program main this PR change
bfs 28750.29 28817.94 +0.24%
editdist 49780.79 49785.21 +0.01%
gameoflife 172360.04 172363.05 +0.00%
hashmap 18499.17 14780.94 -20.10%
histogram 44375.16 44305.37 -0.16%
kmeans 41581.03 41580.60 -0.00%
lexer 53411.26 53108.75 -0.57%
mandelbrot 43455.06 43455.22 +0.00%
merkle 47332.59 47310.58 -0.05%
nbody 39793.90 39793.65 -0.00%
queens 89821.52 89821.39 -0.00%
raytrace 82959.79 82957.98 -0.00%
symreg 50770.05 50770.61 +0.00%
terrain 44571.19 44578.29 +0.02%
tree-bitonic 148715.87 148615.10 -0.07%
tree-matmul 62585.32 62581.35 -0.01%
tree-radix 13827.81 13791.47 -0.26%

By commit: hashmap is -7.8% with the inline take and wrap and -20.1% with the drop; bfs' +0.24% comes with the drop; lexer's -0.57% is the static node read in place; the rest do not move at any step.

The 22 koru kernels (suites/osprey-compute-kernels/bend), instructions: 17 within 0.1%, the worst wordfreq, +0.39% (of 17M); the others:

program main this PR change
binarytrees 1639.20 1609.61 -1.81%
josephus 92.12 92.02 -0.10%
listops 14.01 13.99 -0.18%
textstats 73.25 72.96 -0.40%
wordfreq 17.14 17.21 +0.39%

Cycles on the suite, where a first run suggested a change at equal instructions, by the same 16-placement sweep: hashmap 9.59 to 9.80G on main, 8.90 to 9.09G here (1.07x to 1.10x); nbody and editdist have the same spread on both (their one-run differences, +5% and -20%, were placement); lexer is 8.44 to 8.96G on main (median 8.50G) and 8.67 to 8.80G here (median 8.73G): 0.6% fewer instructions and a median 2.7% more cycles, inside main's own spread over placements. Not explained.

Found on the way

  • A leak, by the memory check on an existing test (tests/io/chan_shared_node, 9 live words): a closure made inside the lent copy of a def captured the lent box as if it were still lent, so it added a second count and never dropped its own. The adversarial programs had no closure in a lent def; lend_flat has one now, and the wrong version is caught. Sibling looked for: every other place a value crosses into a scope that owns it (frames, tasks, the inlined once-called def); none can hold a lent word, since a lent copy is flat.
  • A leak, by review, before any test showed it: an operation whose C names its operand twice (Map.bit) added the count of a lent word twice. lend_flat_twice covers it. Sibling: every template of the C lane goes through the same alias now.
  • A build break: ctr_take is called by the effect sources (effs/audio.c, effs/process_run.c), so its signature stays as it was (a first version gave it a parameter and 30 io tests stopped building).
  • Three regressions on Bend's suite, each fixed before this was opened: kmeans +21% (step 1 without its every-path clause), histogram +116% (step 3 with the decrement inline in term_sink: clang stopped inlining four spins), lexer +2.9% (step 2 as Keep shared-node takes inline on the CPU #2114 has it; see there).
  • Main: ThreadSanitizer reports a race in the scheduler (monk_step's atomic write against a plain read in row_grow) on main as on this branch: seen on lend_flat at 8 threads, in 4 runs of 5 on main. Not looked into.

Fixed after review: a use-after-free with an erased field (commit 5)

Found by another agent's program, reproduced here (tests/rfc/erased_lookahead.bend): a node type with an erased field among its boxed ones, EN{-n: Nat, l: E, b: Bool, k: U32, r: E}, and a lookahead insert. On dd56b9e it dies with a memory fault at -O3 and under -DBEND_POISON; main, the interpreter and the JS lane print 467313141.

Cause. Step 1 rewrites an arm that re-forms its node (arm_self): it opens the arm's field binders and replaces a constructor of exactly those binders by the scrutinee. The rewritten arm's binders are lambdas whose bodies are computed again when they are entered. A live binder is entered through term_open, which hands back the rewritten body; an erased binder is entered by emit_body with a fresh variable (x.f(Var)), and the body computed again from that variable no longer matches the constructor, so the arm came back unrewritten. The scrutinee's uses had been counted on the rewritten arm, so its fields were read through the reference (the node kept whole), and on the path that used the fields instead the node was dropped first: in the reproducer, term_sink of the left child, then its peeked fields stored. With the erased field later in the node the same mismatch crashed the compiler instead (b.n of an unbound use).

It was not slot arithmetic: node slots, peeked fields, the box test and the split all count live fields only, and are right with erased fields (checked below).

Fix. A field binder of a rewritten arm answers the rewritten body whatever variable enters it (+42 ttok, comp.ts 67,588). Only arms the rule rewrites are touched: the C of all 913 earlier tests, the 7 Perceus ports, the 17 benches and the 22 kernels is byte-identical to before the fix, so every number above stands.

Siblings tried, each now a test in tests/rfc/ (a tree type whose node holds the fields in the order given; same, swap, a peek of the key, a lookahead that re-forms the child, the lookahead insert, a flat def lent the tree with an erased parameter first, unique, shared and forked, and dropped):

test fields on dd56b9e fixed
erased_first -e1, l, k, r memory fault ok
erased_middle l, -e1, k, r compiler crash ok
erased_last l, k, r, -e1 compiler crash ok
erased_two -e1, l, -e2, k, r memory fault ok
erased_two_adjacent l, -e1, -e2, r, k compiler crash ok
erased_after_split c, -e1, l, k, r (colour split, then erased) compiler crash ok
erased_before_split -e1, c, l, k, r memory fault ok
erased_record_sum a record with two erased fields among boxes, and a three-armed sum with erased fields, nested memory fault ok
erased_lookahead the reproducer memory fault ok

Each fixed one agrees on the interpreter, the JS lane and C at 1 and 8 threads with poisoned frees, with main's live count (0), and is clean under ASan and TSan (but for main's scheduler report). The fix reverted is caught by all nine. The rule still fires on them (same, the key peek and the lookahead read the node through the reference); a rebuild with a different erased value (renum) is not taken for the original, which is conservative. Step 4's lent copies take the right parameter past an erased one (spin_push_1), and step 3's table counts slots only (EXN has arity 3 for 5 fields).

Why the earlier tests missed it: no program had an erased field in a node the rule reaches.

Rerun on the fixed tree: the native lane against main, the same status on every earlier test; the 9 new tests pass. Four statuses flipped in this run (string_contains and string_ends_with timed out, gfx/clicks and gfx/window passed where main's baseline run had failed them): the machine was at a load of about 30, and the four pass on main and on the fix when run alone. The memory check at 1 and at 8 threads: 913 earlier tests, all the same as main (their C is byte-identical to before the fix); the 17 new tests run through the same check one by one (same output, live count 0). Metal: the 46 tests with a ! call (now with arm_reform_bang) pass with the GPU on, as main's 45 do. JS: the emitted JS of the 913 earlier tests is byte-identical to main's (the JS lane never takes the rewrite).

Checked

  • Native test lane (every tests/**/*.bend with #| lines, built and run with the GPU off, compared with those lines): 890 ok, 22 that differ and 1 that times out on main; the same status on every one of them here, plus the 8 new tests.
  • Memory (every test's C from main and from this branch, built with -DBEND_POISON and a count of live heap words at exit, run at 1 thread and at 8, compared on output, exit code and live count): 913 tests, the same on every one, at both thread counts.
  • Metal: the 45 tests with a ! call, run with the GPU on, main against this branch: 45 pass on both. The new arm_reform_bang test runs steps 1 and 4 on the device.
  • JS: the emitted JS of 913 tests is byte-identical to main's.
  • Adversarial programs (now tests/rfc/arm_reform*.bend, lend_flat*.bend, drop_fall.bend): a node unique on one path and shared on another through the same def; a tree two holders share, one rebuilding a path and the other reading the old tree afterwards; a shared node inspected and then dropped by its other holder, on other threads; rebuilds with two fields swapped, with one field changed, and twice; constants; deep recursion both ways; a spine that is not the last field; the same def called by a caller that keeps its tree and by one that gives it away; the same tree twice in one call; a type shared only in a branch that never runs. Each agrees on the direct run, the JS lane and C at 1 and 8 threads, with main's live count (0), and is clean under -fsanitize=address and -fsanitize=thread, but for a report in main's own scheduler (monk_step against row_grow) that main gives too.
  • Wrong versions, each built and run against those programs: the box-field test off; fields compared out of order; the colour split off by one; the peeked fields owned; the split keeping its first field; a lent box not kept, kept twice, not sealed, or owned; a kept word named twice by an operation; a closure capturing a lent box as lent; a shared node freed by the drop; a count cell not freed; a static node freed; a child skipped; no depth bound; a count dropped twice; a node with two holders taken in place. All 18 are caught; four more only change speed.

Not covered

  • The node is still freed before the call and allocated after it, and so is its count cell: after this PR rbtree-ck does the count operations of a hand-written version (25.2M copies, 49.1M increments, 74.3M decrements) but 126.8M node allocations against 29.4M and 134.4M count cells against 29.4M. By hand edit, keeping the node and its cell across the call is a further -7% instructions on rbtree-ck-unsafe and -19% to -25% on rbtree-unsafe, with no change in cycles that I could measure at this load. That is Build constructors before the calls that fill them (constructor tails and forks) #1810's ground (it builds the constructor before the call, and leaves hot constructors and arms whose result feeds another def alone); this PR does not touch it and the two should compose.
  • An outer node whose fallback row hands it back after a nested pattern failed (case Node{c, Node{Red{}, ..}, k, r}: .. then case t0: t0) is still taken and rebuilt: the rewrite sees a field consumed by the nested match.
  • A split only on the first field, and only when it is an enumeration.
  • A lent parameter only for flat defs that do not loop, and only a single box of a datatype (not an Array, not a pair).
  • The devices keep the walk for drops, and the out-of-line take and wrap.
  • A drop that knows the node's shape (code per datatype): by hand a further -6% instructions on rbtree-ck-unsafe.
  • The count cell as a separate cell (against a count inside the node): a design question, not touched.
  • Not run: the cluster gates (so not gates/test.ts' repeated in-process compiles either), CUDA, Linux, x86, E-cores.

🤖 Generated with Claude Code

nicolas-abril and others added 4 commits November 18, 2026 22:50
…e whole

A constructor that holds exactly the fields its arm bound, in their places,
is the scrutinee (arm_self, as Nat and word matches already do). When every
path of the arm uses it and no path uses a box field for anything else, the
arm reads its word fields through the reference and takes nothing. An arm
that opens with a match on a first field that is an enumeration is split by
that field, read through the reference.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
span_fade and rfc_wrap are inline on the host (the take is bendlang#2114's change,
by Lorenzobattistela) and unchanged on the devices. A constructor with
static nodes that is not hot is read in place, like any uncounted node:
the inlined counted path made each of lexer's takes longer.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
term_sink goes to term_fall on the host: a constructor node is freed at
once, its spine field loops, the others recurse, and a child's count is
dropped inline. Past 64 levels and for anything else it is term_drop's
walk; the devices keep the walk.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
A call to a flat def that does not loop, with an owned box the caller uses
afterwards in a parameter the def owns, calls a copy of the def in which
that parameter is lent: the def adds the count where it owns the box and
does nothing where it only dropped it. The caller seals the box first.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
nicolas-abril added a commit to nicolas-abril/bend that referenced this pull request Nov 19, 2026
…d; the test (on bendlang#2207)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…red fresh

arm_self rewrites an arm that re-forms its node to the scrutinee, matching
the constructor against the arm's own binders. emit_body enters an erased
binder by calling its lambda with a fresh variable rather than through
term_open; the rewritten lambda then ran the rewrite again on that
variable, no longer matched, and gave back the arm with no rewrite, while
the scrutinee's uses had been counted on the rewritten one. The arm read
the node's fields through the reference and dropped the node first: a
use-after-free (an erased field first) or a compiler crash (one later).

A field binder of a rewritten arm now answers the rewritten body whatever
it is entered with. Tests: erased fields first, in the middle, last, two
apart and two together, before and after a split colour, in a record and
a three-armed sum, and the reproducer.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
nicolas-abril added a commit to nicolas-abril/bend that referenced this pull request Nov 20, 2026
…egration, bendlang#2207), report.py, warm-up runs

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant