Skip to content

:strict reference check: one pass over the bytes, no allocation - #145

Merged
mathieu17g merged 3 commits into
mainfrom
strict-reference-check-scan
Sep 10, 2026
Merged

mathieu17g merged 3 commits into
mainfrom
strict-reference-check-scan

Conversation

@mathieu17g

Copy link
Copy Markdown
Collaborator

Closes #142.

parse(…; wellformed = :strict) allocated about seven times per reference, in the Node and FlatNode parses alike. The check matched a regular expression with three groups per reference. Each match cost a RegexMatch and its captures, and a numeric reference a SubString for tryparse besides.

The check now reads the bytes of a token that carries a & and allocates nothing. Each & is found with findnext over the code units. A numeric reference has its digit run accumulated and tested against the XML Char range. A named one is compared byte for byte with the five predefined names, when the "Entity Declared" constraint applies. Only a rejected reference is copied, into its message.

What the check accepts, rejects and ignores is unchanged, message included. The digit run is read on the hexadecimal alphabet in both forms, as the pattern read it, so a decimal form carrying a letter is still rejected. A & that starts neither form is still left unchecked. The test suite pins this with a comparison against the pattern-based check, kept as the specification. It covers every string of one to four tokens over an alphabet that spells each reference form, each edge of the character range, a broken form and multibyte text: 406,900 strings. Each is checked as String and as SubString, names on and off, verdict and message compared. The edges are listed one by one as well. Allocation guards hold the check at zero, and parse(…; wellformed = :strict) at the allocation count of :structural on a document of references.

The decoder and the check now share one reference lexer. unescape resolves a reference through the same helpers, with its own outcomes, and the five names and the characters they denote sit in one place. Its outcomes are unchanged, pinned by the 579,194-string comparison of #144.

Measured

Section (8) of benchmarks/profile.jl, escaped twin of #140, 81,799 references; BenchmarkTools medians, Julia 1.12.7, final commit:

row before after
escaped :strict 89.8 ms (GC 13.5) · 3,241,570 68.9 ms · 2,658,494
escaped :structural (control) 56.8 ms · 2,658,494 56.4 ms · 2,658,494
plain :strict (control) 59.8 ms · 2,526,927 59.1 ms · 2,526,927
text-only :strict (control) 7.05 ms · 18 6.97 ms · 18

The acceptance line of #142 was escaped :strict at the allocation count of escaped :structural.

Per call, on a value of a spreadsheet cell's shape, A & B <tag>: 281 ns and 19 allocations → 17 ns and 0. On a 98-byte text token with one reference: 117 ns and 8 → 8 ns and 0. unescape on the same cell value: 33 → 34 ns, within the run-to-run spread. The shared-strings cells of benchmarks/benchmarks.jl move within noise: Node build 9.93 ms unchanged, normalized write 51.6 → 50.7, entity-heavy sst_load! 19.7 → 19.8.

Docs

  • PERFORMANCE-v0.4.md: Table 7 republished from this campaign, its note dated per table; the sentence under it no longer describes a pattern match per reference. Table 6, benchmarks_results.md and the README table keep the figures of unescape: one pass over the bytes, one allocation per decoded value #144: nothing they measure depends on this change, and the same campaign re-measured them within noise.
  • CHANGELOG.md: a Fixed entry.

Left out

_check_chars_strict, the character-range scan :strict runs on every token, is #143. entities.jl still resolves the character references of an entity's replacement text with its own regular expression, once per declaration, off the per-value path.

@codecov-commenter

codecov-commenter commented Sep 5, 2026 •

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 96.61%. Comparing base (1448084) to head (f17f900).

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #145      +/-   ##
==========================================
+ Coverage   96.57%   96.61%   +0.03%     
==========================================
  Files          15       15              
  Lines        2573     2597      +24     
==========================================
+ Hits         2485     2509      +24     
  Misses         88       88              
Files with missing lines Coverage Δ
src/escape.jl 99.43% <100.00%> (+0.04%) ⬆️
src/parse.jl 99.40% <100.00%> (+0.04%) ⬆️
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

The check ran a three-group pattern over every token carrying a `&`: a
RegexMatch and its captures per reference, a SubString for tryparse per
numeric one, about seven allocations per reference and 583,076 on the
escaped twin of the benchmarks. The check now finds each `&` with
findnext over the code units and reads the bytes that follow: the digit
run of a numeric reference, tested against the XML Char range; the name
of a named one, compared byte for byte with the five predefined names.
What the pattern accepted, rejected or ignored is kept, message
included, pinned by a comparison with the previous check on 406,900
short spans and by allocation guards on the check itself and on parse
at :strict against :structural.

Refs #142.

Assisted-by: Claude (Anthropic)
unescape resolved a reference through its own byte reader and the
`:strict` check through another, each spelling the five predefined
names and the two numeric forms. The helpers now sit in escape.jl and
serve both: `_charref_at` reads a numeric reference, `_predefined_at`
the five names with their `;`, unrolled at compile time from
`_PREDEFINED` so that each comparison reads an immediate, and
`_name_end` delimits any ASCII name for the check alone. Each caller
keeps its own outcome, the decoder keeping as written what it cannot
resolve where the check rejects, and the outcomes of both are
unchanged, pinned by the two exhaustive comparisons of the suite.
unescape stays at 34 ns on a cell's value.

Refs #142.

Assisted-by: Claude (Anthropic)
Table 7 of PERFORMANCE-v0.4.md is republished from one campaign on the
final commit: the escaped twin at :strict goes from 89.8 ms with 13.5 of
GC to 68.9 ms, its ratio to :structural from 1.3× to 1.2×, the two
control rows within the run-to-run spread. The sentence under the table
no longer describes a pattern match per reference, the libxml2 figures
beside it and the :lenient/:structural pair come from the same run, and
the note under the tables dates Table 6 and Table 7 separately, Table 6
keeping the figures of the previous campaign. The CHANGELOG gains its
Fixed entry.

Refs #142.

Assisted-by: Claude (Anthropic)
@mathieu17g
mathieu17g force-pushed the strict-reference-check-scan branch from b898127 to f17f900 Compare September 10, 2026 21:40
@mathieu17g
mathieu17g marked this pull request as ready for review September 10, 2026 22:08
@mathieu17g
mathieu17g merged commit b9c005c into main Sep 10, 2026
13 checks passed
@mathieu17g
mathieu17g deleted the strict-reference-check-scan branch September 10, 2026 22:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

The :strict reference check allocates seven times per reference

2 participants