Parser: Count errors in error_count so binding can skip the error walk - #2217
Conversation
ParseResult#errors collects errors by walking the entire AST (value.recursive_errors) on every call, allocating along the way — even when the template parsed with no errors, which is the overwhelmingly common case. Count errors as they are materialized onto nodes (rb_errors_array_from_c_array) and record the total on the ParseResult as @total_error_count. When it is zero, ParseResult#errors returns the top-level errors directly and skips the full recursive walk entirely.
Node#recursive_errors was `errors + compact_child_nodes.flat_map(&:recursive_errors)`, which allocated an intermediate array (and a compacted child array) at every node. Rewrite it to walk children iteratively into a single shared accumulator, avoiding the per-node throwaway allocations on large trees.
a8da5bc to
ff76eeb
Compare
|
@joelhawksley just to confirm, is the |
There was a problem hiding this comment.
Thanks @joelhawksley!
Similar concern as in #2199, I moved it up a layer too so it lives in libherb and can be shared across all bindings.
error_count is now a real total on parser_options_T, filled in by the generated error helpers and by an exact walk at the end of herb_parse, so the file-scope counter is gone. The walk is needed because a handful of errors get attached directly instead of going through append_*, and without it a template like <% if condition without end %> reported zero errors while the tree held one.
Every binding now carries the count on its parse result and skips the recursive walk when it is zero, so Ruby, JavaScript, WASM, Java and Rust all get it.
I also took your accumulator idea into the TypeScript, Rust and Java node classes, since all three had the same per-node allocation issue.
Please let me know if this still yields the numbers you were seeing in your benchmarks!
Thank you! 🙏🏼
error_count so binding can skip the error walk
|
Re-benchmarked with the actual upstream code on both sides (the earlier numbers were from a port onto an older To answer your earlier question directly: the Methodology
Results
Delta: ~9.75M fewer allocations (−15.7%) and ~1.7s faster (−5.4%) on this corpus. So yes — this still yields the improvement, and the allocation win (which is deterministic run-to-run) holds cleanly on top of #2199. 🎉 This benchmark was produced by Claude Opus 4.8 (GitHub Copilot), acting on behalf of @joelhawksley. |
|
Awesome, thanks for re-checking @joelhawksley! 🙏🏼 Merging this then 🎉 |
Summary
ParseResult#errorscollects errors by walking the entire AST on every call, allocating an intermediate array at every node, even when the template parsed with no errors, which is the overwhelmingly common case. EveryHerb::Enginecompile callsparse_result.errors, so this walk runs once per template.This PR makes the clean-template path free, and does it once for every binding:
libherb, not in a binding.parser_options_Talready carried anerror_countfield for themax_errorscap.herb_parsenow fills it with the exact total and hands it back to the caller, so Ruby, JavaScript, WASM, Java and Rust all get the count instead of Ruby alone.recursive_errorswaserrors + compact_child_nodes.flat_map(&:recursive_errors), allocating throwaway arrays at every node. It now walks children iteratively into a single accumulator, in all four language bindings.Split out of #1872 so it can be reviewed and benchmarked on its own.
Implementation notes
The total comes from one allocation-free visitor walk at the end of
herb_parseinstead of counting as errors are constructed. Six sites attach errors directly instead of going through the generatedappend_*helper, including the Ruby parse errors inanalyze/parse_errors.c, so counting at construction time reported zero errors for a template like<% if condition without end %>while the tree held one. The walk is exact regardless of how an error was attached.herb_parseonly installs its own counter when the caller leaveserror_countNULL. A binding that supplies one reads the total back, and a binding that does not getsniland simply walks, so the fallback is safe by construction.max_errorsnow caps every error type instead of only the two tag-matching sites it reached before, which is what the option has always been documented to do. This is a behavior change. The corpus run reports 129 fewer diagnostics across 36,976 files with no file changing verdict.Benchmark
Measured end-to-end through the
github/githubmonolith'sViewPrecompiler.precompile, which compiles the full template set through the Herb engine (soParseResult#errorsis exercised once per template). Numbers are the full-precompile pass (bin/rails runner), comparing the vendored gem built frommainvs. built with this change:main)≈9.74M fewer allocations (−15.3%) and ≈2.0s faster (−5.8%) across the whole precompile.
Correctness
error_count == 0, the fast path returns the top-level errors and skips the walk.max_errors.error_count == recursive_errors.sizeacross every error path in Ruby, both JavaScript bindings, Java and Rust.max_errors, so the cap silently did nothing there while WASM honoured it. Java seeded the parse result's error list from the document node, which the recursive walk then visited again, double counting every document-level error.cargo +nightly fmt --checkclean.This change was produced by Claude Opus 4.8 (GitHub Copilot), acting on behalf of @joelhawksley.