Skip to content

A call into a tree recursion runs the callee's returning arms itself and jumps straight to its continuation - #2177

Draft
nicolas-abril wants to merge 4 commits into
bendlang:mainfrom
nicolas-abril:opt/call-site-returns
Draft

nicolas-abril wants to merge 4 commits into
bendlang:mainfrom
nicolas-abril:opt/call-site-returns

Conversation

@nicolas-abril

@nicolas-abril nicolas-abril commented Nov 16, 2026 •

Copy link
Copy Markdown
Collaborator

comp.ts is 67,191 ttok, 1,191 over the 66,000 cap (main is 65,966; #2176, which this stacks on, is 66,000; the first commit here 66,774, arms under nested tests +205, the frame id's move +212), so repo-gate fails until that is settled. Two commits were added on 2026-11-16; what they change and cost is in its own section below, and the sections before it describe the first commit as it was measured. Measured on one Mac (M4), not on the cluster: the test and perf gates have not been run. Stacks on #2176: the first commit here is that PR.

What it does

A non-tail call pushes a frame that ends in the id of its continuation and jumps to its callee. When the callee is at a leaf of the recursion, it tests its argument, computes a word, pops the id and jumps back through the segment table. In a tree recursion that is at least half of all calls: eval in symreg walks 63 nodes a tree and 32 of them are Var or Lit.

The call now runs the callee's returning arms itself. After the frame is pushed and the arguments are in the registers, the caller tests them as the callee would; an arm that returns at once runs in place and jumps straight to the continuation the frame names (WL_POPN(1); WL_JMP(FID_k), where the callee had WL_RETN). Any other arm falls through to the call, unchanged.

r0 = _f_0;
r1 = _x_0;
if (!DEVICE && WL_SEQ(FID_eval)) {      // the frame is on the stack
  Term _e_0 = r0;                       // eval's own parameter reads
  u32 _x_0 = r1;
  if (term_aux(_e_0) == CID_Var) {      // eval's own arms
    r0 = _x_0;
    WL_POPN(1); WL_JMP(FID_eval_k1);
  } else if (term_aux(_e_0) == CID_Lit) {
    u32 _v_0 = term_loc(_e_0);
    r0 = _v_0;
    WL_POPN(1); WL_JMP(FID_eval_k1);
  }
}
WL_JMP(FID_eval);
  • The arms are the callee's emitted text. emit_fork leaves a //base callee continuation line at the call; once every unit is final, segs_base replaces it with the callee's parameter reads and the leading chain of its entry segment. The code that runs is the code the callee would have run, on the same registers, so ownership, layouts and frees are the callee's own.
  • Which arms: one that ends in WL_RETN and has no other jump, push, loop exit or return. An arm that calls a spin is left out (clang inlines a spin while it has one call site: with such arms copied, kmeans ran 1.9% and raytrace 3.1% more instructions). An arm of 8 lines or more is left out. The other arms keep their tests with empty bodies, so the chain reads as in the callee.
  • Which callees: a def that calls itself twice on one path (term_calls), the shape whose calls mostly end at once. Without this rule a recursion down a list got the test too, once per element for one hit at the end, and lexer lost 3.1% of its cycles (8.09G to 8.34G, medians of two 16-position sweeps that do not overlap) for no instructions.
  • Where: the host, and only where the frame is on the stack (WL_SEQ, or always after a fork's sequential path). The device compiles the block away.

Results

One M4, GPU off, one thread unless said. Instructions retired and cycles from /usr/bin/time -l, builds run in turn, minimum of 3 to 5 runs. "#2176" is the commit this stacks on.

main #2176 this C twin
symreg instructions 50.77G 42.75G 40.81G 37.96G
symreg cycles (one placement) 12.37G 12.28G 7.76G 9.01G
symreg wall, 8 threads 0.52 s 0.52 s 0.34 s
symreg cycles on an E-core 17.29G 17.13G 12.67G 10.64G

symreg goes from 1.37x behind its twin to ahead of it on a P-core (7.8G to 8.4G cycles across code positions, below, against the twin's 9.01G, which was not swept); on an E-core it is still 1.19x behind.

Placement: the gain is mostly cycles, so the three builds were swept over 16 code positions (nops at the head of work_loop, 4 bytes a step), 3 interleaved rounds, the minimum per position:

build min median max
main 12.19G 12.63G 13.42G
#2176 12.15G 12.43G 12.72G
this 7.80G 8.18G 8.42G

The ranges do not overlap: the slowest position of this build is 3.8G cycles under the fastest of main. Position by position the ratio runs from 1.47x to 1.67x, median 1.55x. Placement alone moves this build by about 8%.

Counters (Instruments' CPU Counters, whole run):

build taken branches returns through the table mispredicted
main 8.13G 1.85G 0.044G
#2176 8.13G 1.85G 0.045G
this 5.95G 0.92G 0.044G
C twin 6.29G 1.62G (ret) 0.021G

Half of the returns are gone, which is the 32 leaves in 63. symreg was never short of prediction (0.02 mispredictions a node) or of cache (no misses): over twelve variants of the bench its cycles follow taken branches (R² 0.95) and not instructions (R² 0.23).

Three small recursive kernels (scaled copies of the fib, hanoi and tak ports in koru-benchmarks' osprey-compute-kernels), instructions / cycles at one placement:

kernel main #2176 this C reference
fib(42) 17.89G / 2.86G 16.65G / 2.74G 13.64G / 2.03G 7.38G / 1.74G
hanoi(29) 15.85G / 2.88G 10.76G / 2.32G 10.22G / 2.08G 6.45G / 2.20G
tak(38, 19, 9) 57.07G / 10.07G 44.92G / 9.75G 36.26G / 6.11G 46.71G / 7.14G

The rest of the suite, against #2176: merkle -0.03% instructions, tree-bitonic -0.01%, tree-matmul -0.17%, tree-radix +0.04%, cycles within the noise; the other 12 emit the same C. queens is not reached: solve's returning arms are nested under an inner test, not arms of its entry chain.

Elsewhere: 52 tests emit different C from #2176. demos/pure_hvm5_mini's C grows 3.2% (72 arms at 42 calls), demos/proof_typed_eval's 7.4% (20 arms at 13 calls), pure_par_sort's 1.3% and pure_par_sum's 0.4%; app_slash_boss_3d and the other 10 demos have no such call. Emitting a demo's C took the same time in single runs (0.1 s to 1.4 s each).

Added 2026-11-16: arms under nested tests, and the frame's id

Two commits on top of the one described above (9b1dc6ec, 74a1c357). Everything in this section is measured against that first commit ("#2177 as it stood", 3394908c), on the same M4, GPU off, one thread.

What changed

  • Arms under nested tests. seg_base now walks the callee's entry as nested blocks instead of its leading chain only: a test below the entry chain is copied with its returning arms. The lines between two tests run a second time in the callee when no arm answers, so a nested test is copied only when those lines just name words (a declaration whose value calls nothing but term_loc, term_aux, term_peek or U32_BIN); a line that takes a node apart (ctr_take), frees, allocates or calls a spin keeps everything below it in the callee.
  • An arm that drops a subterm stays in the callee (term_sink). The drop is a call to term_drop out of line, and it gave the segment the arm was copied to a C frame on every entry (warp_k1 in tree-bitonic gained stp x29, x30): with such arms copied, tree-bitonic ran 10.2% and tree-radix 3.7% more instructions. This also takes such arms away from the entry chain, where the first commit copied them; no bench or kernel had one.
  • The frame's id is stored below the arms. Where a call gets arms, the store of the continuation's id moves after them, so an answered call neither stores nor pops it (WL_JMP(k) where the first commit had WL_POPN(1); WL_JMP(k)). Not in the callee's own entry segment: there the call closes a loop, and with the store after the test clang vectorises the loop's frame pushes (st4.2d in tak, which then ran 5.7% more cycles at every placement for 4.8% fewer instructions). A call that gets no arms is emitted exactly as before.
r0 = _f_13;                              // mul_k1 in tree-matmul
r1 = _f_18;
if (!DEVICE && 1) {
  Term _a_0 = r0;
  Term _b_0 = r1;
  if (term_aux(_a_0) == CID_Lf) {
    u32 _x_0 = term_loc(_a_0);
    if (term_aux(_b_0) == CID_Lf) {      // a second test
      u32 _y_0 = term_loc(_b_0);
      r0 = term_pak(CID_Lf, U32_BIN(_x_0, *, _y_0));
      WL_JMP(FID_mul_k2);                // the id was never stored
    } else {
      ...
    }
  } else {
    u64 _sp_1 = term_peek(e.mem, _a_0);  // lines that only name words
    ...
    if (term_aux(_b_0) == CID_Lf) {
      ...
    }
  }
}
if (1) {
  STK(0) = FID_mul_k2;
  WL_PUSHN(1);
}
WL_JMP(FID_mul);

Results against #2177 as it stood

Instructions retired, and cycles over 16 code positions (3 interleaved rounds, the least per position; least / median / most, in G). Another agent was benchmarking on the machine during these sweeps.

bench instructions cycles, #2177 as it stood cycles, now per position
tree-matmul 62.70G to 46.80G (-25.4%) 7.98 / 8.05 / 8.09 5.86 / 5.91 / 5.98 0.729x to 0.743x, no overlap
queens 84.13G to 71.10G (-15.5%) 21.82 / 22.05 / 22.27 19.84 / 19.89 / 20.08 0.892x to 0.916x, no overlap
tree-radix -0.3% 14.92 / 15.22 / 15.97 14.58 / 14.75 / 14.81 0.924x to 0.988x, no overlap
tree-bitonic -0.29% 20.20 / 20.38 / 20.48 20.08 / 20.21 / 20.37 0.988x to 0.998x, the ranges overlap
symreg -0.08% 7.69 / 8.13 / 8.47 7.71 / 8.00 / 8.39 0.956x to 1.048x, the ranges overlap
merkle -0.02% 18.54 / 18.55 / 18.55 18.54 / 18.55 / 18.55 the same range
the other 11 same C

Counters over the whole run (one run each): tree-matmul 4.72G to 4.13G taken branches and 1.889G to 0.247G indirect ones (87% of its returns through the table are gone); queens 6.54G to 4.52G taken and 0.857G to 0.551G indirect. Mispredictions barely move (19.7M to 16.9M, 574M to 559M).

symreg differs in 9 of its 46 segments (the continuations of eval and esize, and floop): only the frame id's store moved; eval's entry is textually the same.

The koru kernels, scaled as above (pascal 30/15, coins 1200, binarytrees 8000 trees, ackermann 3/12, exprtree 600 trees):

kernel instructions cycles, #2177 as it stood cycles, now per position
tak(38, 19, 9) 36.26G to 32.83G (-9.5%) 6.14 / 6.26 / 6.83 6.14 / 6.27 / 6.84 0.948x to 1.100x, the same range
fib(42) -1.96% 2.026 / 2.030 / 2.047 2.020 / 2.026 / 2.059 the same range
hanoi(29) 0 2.073 / 2.082 / 2.109 2.072 / 2.082 / 2.107 the same range
pascal 20.57G to 23.56G (+14.5%) 3.99 / 4.09 / 4.33 3.77 / 3.89 / 4.18 0.927x to 0.987x
coins 14.81G to 11.23G (-24.2%) 4.20 / 4.27 / 4.35 4.19 / 4.34 / 4.35 0.963x to 1.035x, the same range (main: 3.88 / 3.88 / 3.91)
binarytrees -1.0% 1.513 / 1.535 / 1.666 1.472 / 1.501 / 1.537 0.894x to 0.994x
ackermann 0 1.457 / 1.466 / 1.473 1.454 / 1.462 / 1.472 the same range
exprtree -0.11% 1.484 / 1.497 / 1.510 1.484 / 1.492 / 1.503 the same range

pascal is the cost: binom's frequent returning arm (n == 0, 42% of its calls) calls two spins and stays in the callee, so the copied tests answer only the rare one (k == 0, 8% of calls) and the rest run them twice. It runs 14.5% more instructions for 3% to 7% fewer cycles.

Per widening

widening built instructions what it costs
arms under nested tests, drops left out (9b1dc6ec) yes queens -15.5%, tree-matmul -19.1%, tree-bitonic -0.1%, tree-radix +1.0%; coins -21.5%, pascal +12.3% +205 ttok; C of tree-matmul +13.9%, queens +4.3%
the frame's id below the arms (74a1c357) yes on top: tree-matmul -19.1% to -25.4%, tree-radix +1.0% to -0.3%, tak -9.5%, coins -21.5% to -24.2%, fib -2.0%, binarytrees -1.0% +212 ttok
the same at the callee's own entry measured, left out symreg -2.8% (cycles 0.915x to 1.016x, overlapping) tak +5.7% cycles at every position: clang vectorises the pushes
arms that call a spin, under 12 lines bounded pascal -8.3% where it is now +14.5%, exprtree -6.9%, tree-bitonic -2.2% raytrace +3.1%, kmeans +2.0%, as first measured
arms of 8 to 24 lines bounded kmeans -0.4% (szip), nothing else none seen
arms that drop a subterm measured, left out tree-bitonic +10.2%, tree-radix +3.7%
callees that call themselves once counted of lexer's 591M non-tail calls 11% end at once (gen 3%, ident 22%, num 29%); digitsum 18%, collatz 1% the measured lexer loss stands
mutual recursion, a second level, non-recursive callees counted under 4% of the calls of any bench (tree-bitonic 3.9%, queens 0.8%)

How the calls of each bench end was counted with a profiling build (every non-tail call tagged with its site, every return with its arm). Of the non-tail calls, those ending at once in an arm under a nested test: tree-matmul 86% of 1.91G, queens 36% of 857M, tree-radix 40% of 238M, coins 50%; in an arm that calls a spin: tree-bitonic 45% of 2.34G, pascal 42%, raytrace 50% and mandelbrot 49% (where the arm is the whole leaf's work).

With the current #2175 (opt/nat-accumulate at 2707077e, this branch patched on): fib, pascal and binarytrees are as with #2177 as it stood (same instructions for the first two, -0.4% for the third, cycles in the same ranges). coins is slightly worse: 11.62G to 12.00G instructions and 2.33 / 2.42 / 2.65G to 2.50 / 2.59 / 2.87G cycles (overlapping). The additions pay on queens and tree-matmul, which #2175 does not touch.

The two known problems

Checks for the additions

All local, against #2177 as it stood (3394908c):

  • Native lane: 885 tests print their #| lines on both, 21 differ from them and 1 times out on both; no status differs. The new test is the 886th.
  • C and JS: of 943 programs that build (tests, demos, benches, the 22 kernels) 64 emit different C (50 tests, 8 demos, 6 benches), +0.14% C in total; tree-matmul +14.6%, pure_hvm5_mini +5.1%, queens +4.4%. The JS of all 943 is byte-identical. Emitting all of them took 214.1 s before and 213.5 s now.
  • Memory: the 50 tests whose C differs give the same output, exit code and count of live heap words under -DBEND_POISON, at the default thread count and at 8.
  • Metal: the 48 tests that make a ! call print the same with --gpu on and --gpu off, before and now (the arms are compiled away on the device; the frame id's move is in the device code of 3 of them, fork_leaf_handoff, gpu_mark and the new test).
  • New test: tests/run/call_returns_nested.bend: two trees matched together with arms that free and arms that reuse their node, three tests deep, a Bool then two Nats (the shape of coins), shared trees read by two walks, a word test under a tag test, one def called by an owner and by a reader, forks, a ! call, a comb 3000 deep; and the shapes that must stay: a nested arm that drops a subtree, and one behind lines that take a node apart, also on a tree that a reader and a picker share. Its lines are main's on the C lane (1 and 8 threads), under bun and under bend file.bend. A counting build takes 9,073 returns under a nested test and 20,271 on the entry chain, at 1 and at 8 threads.
  • Wrong versions: 20 deliberately wrong copies of the pass, run on the two tests and a scratch file, plain and with poisoned frees, at 1 and 8 threads. 8 are caught (the lines before a test not checked, or any declaration accepted there; the id popped though never stored; the id never stored; the id stored first and again; the arm returning to the callee; the id stored when the frame is in a task; the callee's own entry not popping). 12 print the same: 7 change only which arms are copied (drops, spins, long arms, lists, one arm per test, the id's move at the callee's own entry or nowhere), 3 guard text no program emits, and 2 are gaps: a ! call given arms, and arms run while the frame is in a task (seq clear). No test reaches a copied arm with seq clear; that condition is the first commit's.

Verdicts

  • Nested tests: keep. It is the part that reaches queens and tree-matmul. Alone it left tree-radix 1.0% up in instructions (its cycles overlap); with the frame id's move no bench of the suite runs more instructions.
  • Drops left out: keep, as a measured rule like the spin one: the frame it forces is in the disassembly.
  • The frame id's move: keep outside the callee's own entry. At the own entry it is a gain on symreg that I could not separate from placement and a loss on tak that I could.
  • Spin arms: not built, worth a follow-up with a rule on the spin's size: it would turn pascal's +14.5% into about -8%, but as it stands it repeats the kmeans and raytrace losses.
  • Long arms, linear callees, mutual recursion, a second level: not worth it on these benches.
  • Running the continuation at the answering site: not built. By hand it takes coins from 4.38G to 2.87G cycles and fib from 2.20G to 2.06G (one position each, fib's for 2.4% more instructions); it copies continuation text into every arm.

Not covered by the additions

  • The cluster gates, CUDA, x86, Linux and tsc, as above.
  • Clang decides two things here that the compiler does not control: whether it merges the copied tests with the callee's when the call is in the callee's own entry (it does in queens, not in tree-radix' merge, which is where the +1.0% was), and whether it vectorises a loop of frame pushes (pascal, tak).
  • The sweeps ran on a machine another agent was benchmarking on.

Checks of the first commit

All local, against main f0cf0a7c and against #2176:

  • Native lane: every test built and run with the GPU off at the default thread count: 883 print their #| lines on main and here, 21 differ from them and 1 times out on both, no status differs.
  • Memory: with -DBEND_POISON and a count of live heap words at exit: the 51 of those 52 tests that build give the same output, exit code and count as main. An earlier version of this branch, which answered the calls of every callee and not only of tree-shaped ones (so a superset of these calls), gave the same on all 905 tests that build.
  • The new path runs: a build that counts the returns taken at a call. The new test takes 12,872 of them; a larger copy 250,000. Both print main's lines at 1, 2 and 8 threads and pass the memory check at 1 and 8.
  • New test: tests/run/call_returns.bend: a returning arm that frees the node it owns, one that allocates, one that drops another owned parameter, a three-word result, a Nat match two levels deep, a returning arm written after the calling one, a shared tree read by two walks, a forking def and forks over fork-free ones; and the shapes that must stay as they were (a recursion down a list, two defs that call each other). Its lines are main's; it prints them on the C lane (1 and 8 threads), under bun and under bend file.bend.
  • JS lane: the JS of all 975 programs that emit it is byte-identical to main's (compile_book is the C emitter).
  • Metal: the 47 tests that make a ! call print the same with --gpu on on main and here; the block is under !DEVICE, so the device compiles it away.
  • repo.ts fails only on the ttok cap.

Not covered

  • Cluster gates and CUDA have not run. x86 and Linux were not measured.
  • tsc was not run.
  • Arms that call a spin stay in the callee (bounded above).
  • Mutual recursion (a tree walked by two defs that call each other) is not recognised as a tree.
  • Tail calls have no frame to answer and are unchanged.
  • The frame's id is still stored before an arm answers a call made from the callee's own entry (see above).

Cost

+774 ttok over #2176 for the first commit, +417 for the two added (+1,225 over main).

Where this came from

symreg's gap to its twin was put down to branch prediction. The counters say otherwise: 0.044G mispredictions in 12.4G cycles. What the cycles follow is taken branches, about 1.2 cycles each, and a leaf reached by a call costs four or five of them (the jump in, the tag tests, the return through the table). Answering the leaf at the call leaves one or two. Real C calls were bounded earlier at 1.14x for symreg; this gives 1.55x inside the segment machinery.

🤖 Generated with Claude Code

nicolas-abril and others added 2 commits November 15, 2026 23:44
A segment that reaches no fork tested seq before each frame it pushed and
before each call, and carried the task path it never takes: FID_ENTER and
FID_EXIT set seq for a fork-free task, and a forking segment hands a
fork-free callee to a task of its own. WL_SEQ(F) is seq, or on the host
true for a fork-free segment, so clang drops the test, the task path and
the frame the path's calls forced on every entry.

The one such segment that ran with seq clear is a closure body entered
through a Clo~apply that may fork; it now keeps its frames on the stack
above the task's frame, as a fork-free task does.

symreg -15.8% instructions, queens -6.3%, lexer -2.8% (one thread, M4).

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
A non-tail call pushes a frame and jumps to its callee; when the callee's
match takes an arm that returns at once, the callee pops the frame's id
and jumps back through the segment table. In a tree recursion that is at
least half of the calls. The call now carries the callee's returning
arms: after the frame is pushed and the arguments are in the registers,
an arm that neither calls nor jumps runs in place and jumps straight to
the continuation the frame names. Any other arm falls through to the
call as before.

segs_base copies the arms from the callee's emitted entry segment, so
they run the code the callee would have run on the same registers. Only
for a callee that calls itself twice on one path, on the host, where the
frame is on the stack.

On top of the fork-free sequential path, one thread, M4: symreg -4.5%
instructions and 12.43G to 8.18G cycles (medians of a 16-position sweep;
its C twin runs 9.01G); fib(42) 2.74G to 2.03G cycles, tak 9.75G to 6.11G.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
nicolas-abril and others added 2 commits November 16, 2026 20:42
…s too

seg_base walks the callee's entry as blocks: a test below the entry chain is
copied with its returning arms when the lines before it only name words (the
callee runs them again when no arm answers). An arm that drops a subterm
stays in the callee: the call to term_drop gave the segment it was copied to
a C frame on every entry.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Where a call gets its callee's returning arms, the store of the
continuation's id moves below them: an arm that answers jumps to the
continuation with the frame as a return would leave it. Not in the callee's
own entry, where the call closes a loop that clang then vectorises (tak ran
5.7% more cycles), and a call that gets no arms is emitted as before.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
nicolas-abril added a commit to nicolas-abril/bend that referenced this pull request Nov 20, 2026
As for bendlang#2177's arms: a continuation copied to the call site that calls a
spin gives the spin a second call site, and clang stops inlining it.
cfold's cfold_k3 put spin_cfold_add and spin_cfold_mul out of line (+5%
instructions); tak and deriv lose their small gains with the rule.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant