Release 0.1.12: skip alternatives that cannot begin here (first-character BRANCH)

CLAUDE.md

@@ -57,7 +57,7 @@ One compiler, one program format, one evaluator. There is no fallback engine and
 
 **Pattern text → program.** `compiler::compile` parses into a caller-owned arena of `Node`/`Range` scratch (indices only, no source pointers survive) and emits a validated immutable `u32` program. `compiler::prepare` splits the same pipeline so the host can drop its pattern borrow before allocating final output: `Prepared::required_words()` gives the exact size, `emit` consumes the plan. Emission walks the topologically ordered arena iteratively — no native-stack recursion on long concatenations, no per-count expansion of `a{n}`. Submodules under `src/compiler/` add optimizer analyses (`admission`, `candidate`, `classes`, `repetition`) and grammar corners (`escapes` for Annex B, `names`).
 
-**Program format** (`src/program.rs`). One relocatable buffer: header, three words per instruction, class-table pairs, eight-word repeat records, a filter section for properties of strings, optional packed name section. Everything is relative offsets/indices — never a host pointer. `Program::from_words` revalidates opcodes, operand ranges, property IDs, branch/table bounds and name/capture lists under a work budget, so a relocated or re-read buffer is always checked. `HEADER` and `VERSION` in `src/program.rs` are authoritative (currently 11 words / version 13); older versions are rejected outright, and a few docs still quote an earlier number. Word 9 (leading-literal run) is *re-derived* from the instructions during validation rather than trusted — hint words that can't be re-derived (words 7/8) are only allowed to be conservative, never to supply capture semantics.
+**Program format** (`src/program.rs`). One relocatable buffer: header, three words per instruction, class-table pairs, eight-word repeat records, a filter section for properties of strings, optional packed name section. Everything is relative offsets/indices — never a host pointer. `Program::from_words` revalidates opcodes, operand ranges, property IDs, branch/table bounds and name/capture lists under a work budget, so a relocated or re-read buffer is always checked. `HEADER` and `VERSION` in `src/program.rs` are authoritative (currently 11 words / version 14); older versions are rejected outright, and a few docs still quote an earlier number. Word 9 (leading-literal run) is *re-derived* from the instructions during validation rather than trusted — hint words that can't be re-derived (words 7/8) are only allowed to be conservative, never to supply capture semantics.
 
 **Subject access** (`src/input.rs`). `Input` borrows original storage — `&str`, generalized WTF-8 bytes (lone and separately-encoded surrogates accepted), or host `&[u16]`. **Never build a UTF-16 conversion buffer or copy the subject**; that is the core project constraint. All public positions count UTF-16 units, including positions between surrogate halves. ASCII byte storage gets a distinct internal representation with O(1) seeks; non-ASCII byte cursors pack a half-pair flag into the high bit of the byte offset.
 

Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "perex"
-version = "0.1.11"
+version = "0.1.12"
 edition = "2024"
 rust-version = "1.88"
 description = "An ECMAScript regex engine designed for explicit host memory ownership"

README.md

@@ -177,6 +177,8 @@ Case equivalence uses generated Unicode 17.0.0 data under the [Unicode License V
 
 [Test262 pattern conformance](docs/conformance.md) compares every pattern harvested from the suite's regular-expression tests against Node, for syntax acceptance and complete match answers. There is no remaining gap: 48,718 cases compared, nothing unsupported and no difference, on the harvest recorded there.
 
+[Branches](docs/branch.md) lay alternation out flat and skip an alternative, and its backtracking frame, when the next character cannot begin it, by a first-character set re-derived from the instructions during validation.
+
 [Leading literal starts](docs/leading.md) reject an impossible start with a byte comparison instead of an initialized trial. The claim is re-derived from the instructions during program validation, so it cannot disagree with them.
 
 [Required-text admission](docs/admission.md) also bounds where a match can begin when the condition is one the match itself consumes, so text present only before every possible start stops the search instead of retrying it.

docs/branch.md

@@ -0,0 +1,132 @@
+# Branches
+
+An alternation that fails on its first alternative used to pay for every
+other one in turn: a frame popped, a rollback, a dispatcher round trip, and the
+alternative's first instruction executed only to fail. Claude Code's emoji
+pattern, an alternation of eleven top-level alternatives and about two hundred
+nested ones, cost 6,664 instructions per search against the one-character
+subject `"1"`; V8's whole `test` call costs about 240. Format 14 changes three
+things, each a property of the program alone, and keeps the one evaluator.
+
+## Flat layout
+
+`a|b|c` parses as `(a|b)|c`. Emitted as parsed, the first alternative was
+entered under one live frame per alternative, and a success left through one
+jump per nesting level. Ordered choice is associative, so the emitter now lays
+a left-nested chain out flat:
+
+```
+SPLIT  → a, R1     a   JUMP end
+R1: SPLIT → b, R2  b   JUMP end
+R2: c
+end:
+```
+
+Each alternative but the last sits behind its own branch, whose frame resumes
+at the next alternative; each jumps straight to the end. The size is the same
+— two instructions per alternation either way — and the order in which
+alternatives are tried, and so every answer, is unchanged. The left spine's
+inner alternation nodes become no instruction.
+
+## First-character sets
+
+A new instruction, `BRANCH` (opcode 31), is a `SPLIT` whose first arm begins
+at the next instruction and whose operand `a` is the set of characters that arm
+can consume first. When the character the arm would read — in the direction the
+code runs, so backwards inside a lookbehind — is absent or outside the set, the
+arm cannot match: trying it would fail and resume at `b` with nothing changed.
+The evaluator goes to `b` directly, without a frame. When `b` is itself a
+`BRANCH`, it is decided against the same character in the same step, so a
+chain of alternatives that cannot begin here costs one read and one comparison
+each. Each branch passed over is charged one work unit, exactly as executing it
+would be, and the chain stops at the quantum the next instruction would have
+been given, so totals do not depend on how a search is paused.
+
+The set is a 32-bit mask of character buckets. ASCII characters fall in 24
+buckets by their low five bits, so both cases of a letter share one; every
+other character falls in one of 8 buckets by its high bits, which separates
+surrogate halves, the symbol blocks and the CJK blocks. It is derived from the
+emitted instructions by `derive_first_mask`: from the arm's first instruction,
+capture `SAVE`s and position assertions are passed over (dropping a condition
+only widens the set), jumps are followed, and both arms of a nested branch are
+taken; each path ends at its first consuming instruction, which contributes
+its characters. A path that can succeed without consuming, or that reaches
+anything else — an assertion body, a repeat, a backreference, a property, a
+complement, a folded class, a folded non-ASCII character — makes the set
+unknown, and the branch stays a `SPLIT`. A folded ASCII letter under `u` also
+admits every non-ASCII bucket, for U+212A and U+017F. The walk inspects at most
+32 instructions and eight pending paths.
+
+`Program::from_words` derives every `BRANCH` mask again and rejects a program
+whose operand differs, or whose mask admits everything. So, like the leading
+run, a branch's set cannot claim less than its arm can consume.
+
+## Start-anchored claim and start registers
+
+The claim that every match begins at the subject's start was derived at the
+start of every search; on the nested emoji alternation that walk was 313 of the
+654 instructions a miss on a letter cost. It is now stored as word 7's bit 24
+and re-derived on validation; see [candidate starts](candidate.md#start-anchored-starts).
+
+A trial cleared every register before it began: two per capture and two per
+repeat. The emoji pattern has 194 repeats, one per `?`, so every start cleared
+390 registers and charged 390 work units for it. A repeat's registers are
+written by its `REPEAT_INIT`, the only way into the repeat, before anything in
+it reads them, and a frame or undo entry that restores an older value restores
+it to a point before that write. A trial now clears only the capture registers.
+
+## Measurement
+
+Instructions per search (`instructions:u`, difference between 200,000 searches
+and none) of `Search::run` over a counted binding at a quantum of 4096, as a
+host calls it per `test`, against 0.1.11 in the same harness. "cc mix" cycles
+the subjects `t o k e n 1 7` and a space, the graphemes string-width tests in
+Claude Code's streamed output.
+
+| Case | 0.1.11 | 0.1.12 | Change |
+|---|---:|---:|---:|
+| `/^\p{DI}$/u`, cc mix | 432 | 391 | −9.5% |
+| `/^\p{DI}$/u` on 中 | 1,505 | 1,459 | −3.1% |
+| emoji `/g`, cc mix | 2,212 | 946 | −57.2% |
+| emoji on `t` | 728 | 406 | −44.2% |
+| emoji on `1` | 6,664 | 2,569 | −61.4% |
+| emoji on 中 | 6,741 | 1,637 | −75.7% |
+| emoji on 😀, a match | 11,014 | 5,638 | −48.8% |
+| `/ab\|cd\|ef\|gh/` on `xxgh` | 3,183 | 2,124 | −33.3% |
+| `/(foo\|bar\|baz)=(\d+)/` on `a=1&baz=42` | 4,241 | 3,819 | −10.0% |
+| `/a/` on `t` | 446 | 406 | −9.0% |
+| `/a/` on `a`, a match | 1,206 | 1,162 | −3.6% |
+| `/qq_[0-9]+/` on `record_12345` | 498 | 458 | −8.0% |
+| `/\w+/g` on `foo bar` | 2,231 | 2,189 | −1.9% |
+| `/^[a-z]+_[0-9]+$/`, a match | 3,041 | 2,993 | −1.6% |
+| `/\s+$/` on a trailing run | 4,143 | 4,095 | −1.2% |
+
+No case measured slower. What remains is mostly the fixed cost of a search
+that enters the evaluator: about 1,100 instructions to match `/a/` against
+`"a"`, spread over a dozen phase round trips (admission, the start scan, the
+seek, clearing, the trial, validation) of 30 to 100 instructions each.
+
+## Verification
+
+`tests/branch.rs` compares 36 patterns against the same patterns with every
+alternative, at every depth, written between empty lookaheads — whose arms
+begin with an assertion, so no branch carries a set and every arm is tried —
+from every start including past the end, over ASCII, folded, non-ASCII,
+astral and keycap subjects: alternations of literals, classes, captures and
+named groups, lookbehind and negative lookbehind alternatives, folding with
+U+212A and U+017F under `i` and `iu`, empty and end-asserting alternatives,
+backreferences, repeated alternations and sticky searches. It checks the flat
+layout and source order, that repeat registers holding arbitrary values do not
+change any answer, and that validation rejects every single-bit change to a
+mask, a mask admitting everything, either flip of the anchored bit and a
+reserved bit above it. Six injected faults are each caught by the suite:
+non-ASCII characters bucketed as ASCII, `REPEAT_INIT` leaving its position
+register unwritten, a folded non-ASCII character claiming only its own bucket,
+an arm that can match empty contributing nothing, a chain resuming one
+instruction past the next alternative, and a trial clearing no capture
+register. The engine differential (plain, one- and seventeen-unit resumption
+with relocation and growth, from positions, decided in one call), the Test262
+pattern corpus, and the input, candidate, classes, names, legacy, admission,
+lookbehind, run-skip, repetition, modifiers, bound, end-candidate, sets,
+sequences, Unicode-sets, casefold, properties and atom-filter harnesses, and
+the native tier's check, report no differences.

docs/candidate.md

@@ -35,6 +35,8 @@ Program format 10 has nine header words, retaining the first-character descripto
 no ASCII start is possible, or `2 | (lo << 8) | (hi << 16)` for an inclusive
 interval with `0 <= lo <= hi <= 127`. Reserved bits, malformed bounds and old
 format versions are rejected. The additional word costs four used program bytes.
+Since format 14 the descriptor occupies word 7's low 24 bits; bit 24 holds the
+validated start-anchored claim and the bits above it are reserved.
 Bindings check the full eleven-word header when reacquiring a view. AOT and runtime
 programs must use this same format and its version checks.
 
@@ -144,12 +146,10 @@ the 256 KiB literal lookbehind, which measured between 0 and 2.9 percent slower
 across five runs of 21 rounds each: its remainder is too long to scan, and it
 pays only for being told so.
 
-The claim that a program is anchored at the subject's start is derived at the
+The claim that a program is anchored at the subject's start was derived at the
 start of every search, and the same measurement priced that derivation at
-1.3 ns of the empty search. It now decides the common case — a consuming
-instruction straight after the entry `SAVE`s — before building its branch stack.
-Without that, the misses were about 1 ns slower and the literal hits up to
-1 ns slower; `[^0-9]+!` was 0.8 percent slower than before rather than 1.1.
+1.3 ns of the empty search. Format 14 stores it instead, as a validated word 7
+flag; see [start-anchored starts](#start-anchored-starts).
 
 ### Checks
 
@@ -273,14 +273,20 @@ where one attempt charges a handful. Perry's `test` and `exec` calls pay a
 search per call, so a failing anchored pattern paid it on every call (Perry
 issue #10166).
 
-The claim is derived from the instructions rather than stored, like the leading
-run, so no program word can assert it falsely. From instruction zero, capture
+The claim is derived from the instructions by the compiler and stored as word
+7's bit 24, above the descriptor; `Program::from_words` derives it again and
+rejects a program whose bit disagrees, as it does for the leading run, so no
+program word can assert it falsely. Format 13 derived it at the start of every
+search instead, which for an alternation of ten nested branches walked the
+branch stack to its limit each time: 313 of the 654 instructions a miss of
+Claude Code's emoji pattern cost. A search now reads the bit. From instruction zero, capture
 bookkeeping is skipped and a `SPLIT` is followed into both branches; every path
 must reach `START`. Anything first that could consume or match elsewhere — a
 character, a class, a repeat, an assertion, a jump, or `^` under `m` — disables
 it, so `^a|b`, `(?:^)?a`, `(?:^a)+`, `(?<=^)a` and `(?m)^a` are searched as
-before. The walk inspects at most 128 instructions and eight pending branches,
-once per search, on entering admission.
+before. A `BRANCH` is followed like a `SPLIT`. The walk inspects at most 128
+instructions and eight pending branches, once per compilation and once per
+validation.
 
 A start-anchored search then behaves as a sticky one does for choosing starts:
 the candidate scan examines only its first position, and no later start is

docs/engine.md

@@ -20,7 +20,7 @@ Unicode-mode [character properties](properties.md), including general categories
 
 `compile` receives a borrowed pattern, caller-owned `Node`/`Range` scratch, final `u32` storage and a work budget. Nodes contain indices only. The emitter walks the topologically ordered arena iteratively: a long concatenation does not recurse on the native stack, and a counted repetition is not expanded into one program copy per count. Group parsing has an explicit nesting limit of 128; sizes/counters use checked arithmetic and u32 limits. These remain implementation limits, not claims that deeper/larger valid expressions are invalid JavaScript.
 
-The output is one immutable relocatable buffer: an eleven-word header, three words per instruction, class-table pairs, eight-word repetition records, a filter section for the [properties of strings](sets.md), and an optional [name section](names.md). A class-table pair is a literal interval or shared Unicode property reference/complement. Capture indices are u32, with tests above 254 groups. The native-u32 format needs four-byte alignment and is versioned but unstable; version 13 adds the properties of strings — an instruction naming a shared set of sequences and the filter an operator left on it — retaining version 10's final-character descriptor for strict end-anchored patterns, version 9's [sorted literal classes](classes.md), version 8's candidate-start descriptor and version 7's consuming-atom repeat records and lexical flag instructions. Older versions are rejected; this is not cross-endian serialization. `Program::from_words` validates all opcodes, operand ranges, property IDs/modes and branch/table bounds, packed identifier names and capture-index lists under a work budget. It also validates the exact five-instruction shape and capture exclusion for optimized atom repeats. It does not prove arbitrary hand-authored bytecode has compiler-generated control structure or that a candidate/admission hint is a semantic consequence of its instructions; execution can reject invalid control/capture states and is always work-limited.
+The output is one immutable relocatable buffer: an eleven-word header, three words per instruction, class-table pairs, eight-word repetition records, a filter section for the [properties of strings](sets.md), and an optional [name section](names.md). A class-table pair is a literal interval or shared Unicode property reference/complement. Capture indices are u32, with tests above 254 groups. The native-u32 format needs four-byte alignment and is versioned but unstable; version 14 adds the first-character branch instruction and the stored start-anchored claim described in [branches](branch.md), retaining version 13's properties of strings — an instruction naming a shared set of sequences and the filter an operator left on it — and version 10's final-character descriptor for strict end-anchored patterns, version 9's [sorted literal classes](classes.md), version 8's candidate-start descriptor and version 7's consuming-atom repeat records and lexical flag instructions. Older versions are rejected; this is not cross-endian serialization. `Program::from_words` validates all opcodes, operand ranges, property IDs/modes and branch/table bounds, packed identifier names and capture-index lists under a work budget. It also validates the exact five-instruction shape and capture exclusion for optimized atom repeats. It does not prove arbitrary hand-authored bytecode has compiler-generated control structure or that a candidate/admission hint is a semantic consequence of its instructions; execution can reject invalid control/capture states and is always work-limited.
 
 [Required-text admission](admission.md) can reject a long search when a compiler-proved literal/class condition is absent from the original subject. It uses existing instructions and header bits, without increasing program size. Presence still executes the same VM for complete answers.
 

docs/leading.md

@@ -23,7 +23,7 @@ because word 7 already carries that claim.
 ASCII bytes cannot occur inside a multibyte UTF-8/WTF-8 encoding, so a run of
 ASCII characters is also a claim about original bytes.
 
-Program format 13 has eleven header words. Word 9's low byte holds the run's
+Program format 14 has eleven header words. Word 9's low byte holds the run's
 character count, or zero when disabled; its top bit is the separate
 forward-admission claim described in [admission](admission.md). The word costs
 four used program bytes, and the two claims share it rather than adding another.