Skip to content

String.contains tests a position from the node's fields and walks with a prefix def of its own, so a search takes neither string apart - #2196

Merged
nicolas-abril merged 3 commits into
bendlang:mainfrom
nicolas-abril:opt/string-contains-borrow
Nov 18, 2026
Merged

nicolas-abril merged 3 commits into
bendlang:mainfrom
nicolas-abril:opt/string-contains-borrow

Conversation

@nicolas-abril

@nicolas-abril nicolas-abril commented Nov 17, 2026 •

Copy link
Copy Markdown
Collaborator

A Base change only: comp.ts is untouched (65,981 ttok). base.bend goes from 33,694 to 33,847 ttok; it has no cap in gates/repo.ts, and repo.ts passes 56 / 56. Measured on one Mac (M4), not on the cluster. Two commits: the first kept a second copy of String.starts_with's walk (two defs and a law); the second replaces it with one def, described here.

The problem

def String.contains(s, p):
  match s:
    case SNil{}:
      String.is_empty(p)
    case SCon{h, t}:
      String.contains.if(t, p, String.starts_with(SCon{h, t}, p))

The arm rebuilds the node it has just matched to hand it to String.starts_with. A rebuilt node is a new value, so on the C lane String.contains owns its text. At every position it takes the node apart, counts the tail (term_keep) and the pattern, allocates a node (when the text is a literal, which has no spare to reuse) and a count cell for its tail (rfc_seal), and String.starts_with.go takes that copy apart again and drops it. A text the caller keeps is first shared, so every cell of it ends up behind a count cell.

On textstats (a koru kernel: 200,000 words scored with String.length, String.contains(w, "o") and String.starts_with(w, "t")), String.contains is 75% of the instructions.

What changes

def String.contains.pre(p: String, s: String, same: Bool) -> Bool:
  match p s same:
    case _ _ False{}:
      False{}
    case SNil{} _ True{}:
      True{}
    case SCon{y, yt} SNil{} True{}:
      False{}
    case SCon{y, yt} SCon{x, xt} True{}:
      String.contains.pre(yt, xt, Char.is_eq(x, y))

def String.contains.at(p: String, h: Char, t: String) -> Bool:
  match p:
    case SNil{}:
      True{}
    case SCon{y, yt}:
      String.contains.pre(yt, t, Char.is_eq(h, y))

def String.contains(s, p):
  match s:
    case SNil{}:
      String.is_empty(p)
    case SCon{h, t}:
      String.contains.if(t, p, String.contains.at(p, h, t))
  • String.contains.at tests a position from the node's two fields, so nothing is rebuilt.
  • String.contains.pre is the prefix walk, carrying whether every char so far was the same. It is one def whose only call to itself is a tail call, so it needs no law and the C lane runs it as a loop on main. Only String.contains reaches it, and String.contains lends it both strings at every position (it reads them again afterwards), so it borrows them in every program.

With that, the walk in String.contains only peeks: no allocation, no count, no free. When the caller does not keep the text, String.contains still owns it and frees each node as it passes.

The JS lane keeps its intrinsic (string_contains), so its output is unchanged.

Why not String.starts_with itself. The compiler gives a def one ownership per parameter for the whole program. String.starts_with(s, p) is a tail call into String.starts_with.go, so as soon as one caller hands String.starts_with a string it owns, .go owns its strings everywhere, and a search sharing that walk has to count each tail before lending it. Three simpler spellings, measured below, do not keep the gain: passing s itself to String.starts_with (both parameters are already +), calling String.starts_with.go.if from .at, and matching the pattern in String.contains itself.

Measured

One M4, one thread, GPU off, cc -O3, /usr/bin/time -l, against main 9c586f09. Instructions retired:

program main first commit this PR
textstats (koru kernel, literal words) 205.6M 97.6M 73.6M -64.2%
200 searches for an absent 3-char pattern in a 24,576-char heap string the caller keeps 3,419.3M 696.1M 384.7M -88.7%
the same, in a program where another call hands String.starts_with a built string 5,596.4M 714.0M 402.5M -92.8%
200 searches, each in a 3,072-char string built for it 441.4M 243.4M 210.3M -52.4%

Cycles for textstats, the least of 15 interleaved runs at one placement, on a loaded machine (load average about 8): 35.6M on main, 20.9M for the first commit, 17.4M here. The first commit was swept over 16 placements (30.52M / 31.38M / 32.36M on main, 17.90M / 18.55M / 19.77M); this one was not.

The simpler spellings that do not keep the gain (instructions; textstats / kept string / kept string with String.starts_with owning elsewhere / built string):

spelling
main 205.6M 3,419.3M 5,596.4M 441.4M
String.starts_with(s, p) on s itself 205.6M 3,419.2M 5,596.4M 441.4M
.at calling String.starts_with.go.if (the walk shared) 124.1M 696.1M 5,206.4M 243.4M
the pattern matched in String.contains, no .at 109.4M 2,331.2M 2,349.1M 451.1M
.pre called on s itself, no .at 179.3M 3,769.7M 3,787.4M 485.0M
this PR 73.6M 384.7M 402.5M 210.3M

With #2188. On a branch that merges the open optimisation drafts (opt/integration, where #2188 runs a tail cycle of two defs as one loop), the first commit's pair of defs is a loop too, and it is level or better than the flag: textstats 44.6M for the pair and 45.0M for this, the kept string 282.6M and 384.2M, the built string 187.4M and 196.9M (without either, 172.7M, 2,988.8M and 372.5M). The flag costs a step on a mismatch: .pre peeks at both strings before it reads the flag, where the pair's .if read the flag first. Parameters are matched in order and the first that changes must shrink, so the flag cannot come first.

#2148 and #2134, the compiler-side drafts on borrowing, leave these programs unchanged on their own bases.

Reach

Of the programs under tests, demos and bench/runtime, eight mention String.contains; six of them emit different C (base/string_contains, base/string_kit, gfx/fullscreen_input, io/args, io/args_bytes, run/string_contains_long). For the first commit all 945 programs that emit C were compared: 938 the same as main, among them all 17 benches and every demo that builds. Nothing else in Base calls String.contains.

Checks

Against main, locally, on the second commit:

  • Native lane (every tests/**/*.bend with #| lines built and run with the GPU off): 886 pass on main, 887 here; no status differs over the 1,874 tests on both, and the new test passes.
  • Memory: the six changed tests, built with -DBEND_POISON and a count of live heap words at exit: same output, exit code and live count as main on each. The new test ends with 0 live words at one thread and at eight.
  • JS lane: the emitted JS of the programs that mention String.contains is identical to main's.
  • Check and interpreted run (bend file.bend) on the seven tests that mention it: same output as main. bend bend2/base.bend --check-only: ALL PROOFS CHECK.
  • A value main (normalized by the checker, so Base's def runs, not the intrinsic) over 112 searches: the same string as on main.
  • tests/base/string_contains.bend: its unfold law states the def's first step, so it names String.contains.at now. A second law is new: an empty pattern is found at the first position of a text whose tail is unknown (it holds on main too).
  • New tests/run/string_contains_lent.bend: texts built at run time, so they sit on the heap. A text and a pattern searched twice and read whole afterwards; a tail that shares its cells with a kept text, searched both ways; a text built for the search; a search as a tail call; the strings of a kept list; empty texts and patterns; a pattern longer than the text; a match at the very end; an overlapping near-match (aaab in aaaa…); chars past U+FFFF and past Unicode; and one call that hands String.starts_with a built string. Same output from bend file.bend, the JS lane and C.
  • Wrong versions of the two new defs, seven: a false flag answering true, a text that ends first answering true, a spent pattern answering false (each caught by both tests' output on the C lane); the walk comparing a char with itself, the first char not compared, the walk not advancing the text (each refused by the checker); an empty pattern refused at a position (caught by the new law).

Not covered

  • The cluster gates, Metal, CUDA, Linux and x86 have not been run.
  • A proof that related String.contains to String.starts_with on an unknown pattern by computation alone no longer goes through: the two walks are equal, but under different names. No test or demo has one.
  • String.starts_with itself is as on main: it owns its strings when a caller hands them over.
  • With Defs that tail-call each other in a cycle run as one loop: a head's spin holds the other defs' bodies #2188 the flag is slower than the first commit's pair on a long search that misses (above).

🤖 Generated with Claude Code

nicolas-abril and others added 2 commits November 17, 2026 15:38
…nd walks with its own prefix def, so a search takes no node apart

String.contains rebuilt the node it had just matched to hand it to
String.starts_with, so the C lane owned the text: at every position it
counted the tail, allocated a node and a count cell, and
String.starts_with took the copy apart again.

String.contains.at tests a position from the node's two fields, and the
walk under it is String.contains.pre, String.starts_with.go under a name
only String.contains reaches. String.contains lends it both strings at
every position, so it borrows them in every program, whatever the
program's other calls make String.starts_with do with its own.

The JS lane keeps its intrinsic. tests/base/string_contains.bend's
unfolding law names the new helper; tests/run/string_contains_lent.bend
searches kept, shared, built and handed-over texts and patterns and reads
what the caller kept afterwards.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The walk String.contains keeps for itself was a second copy of
String.starts_with.go and its .if helper, with a law so the two could
call each other. One def that calls itself and carries "every char so
far was the same" does the same job: no law, no second helper, and a def
whose only self call is a tail call is a loop on the C lane, where the
pair was a cycle of two segments.

Instructions, one thread: textstats 97.6M to 73.6M (main 205.6M); 200
searches in a kept 24,576-char string 696M to 385M (main 3,419M); the
same with String.starts_with owning elsewhere 714M to 403M (main
5,596M); 200 searches each in a string built for it 243M to 210M (main
441M).

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@nicolas-abril nicolas-abril changed the title String.contains tests a position from the node's fields and walks with its own prefix def, so a search takes neither string apart String.contains tests a position from the node's fields and walks with a prefix def of its own, so a search takes neither string apart Nov 17, 2026
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@nicolas-abril
nicolas-abril marked this pull request as ready for review November 18, 2026 14:52
@nicolas-abril
nicolas-abril merged commit 2196844 into bendlang:main Nov 18, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant