Skip to content

Avoid quadratic anchor lookup in large documents - #363

Open
adenzhou1350 wants to merge 1 commit into
yaml:masterfrom
adenzhou1350:perf/document-local-anchor-index
Open

adenzhou1350 wants to merge 1 commit into
yaml:masterfrom
adenzhou1350:perf/document-local-anchor-index

Conversation

@adenzhou1350

Copy link
Copy Markdown

yaml_parser_register_anchor() and yaml_parser_load_alias() repeatedly scan the aliases vector, making documents with many anchors expensive to compose. This keeps the existing linear path through 64 anchors and builds a document-local AVL index when the 65th anchor is registered.

The private index stores integer offsets into the aliases vector, so vector reallocations do not invalidate it. It is freed on both document success and failure. Public headers, parser layouts and ABI are unchanged; duplicate-anchor marks, undefined-alias errors, cycles and document-local reset retain their existing behavior.

Fixes #359.

Validation on Linux/GCC 12.2:

  • Default CMake Release (-O3 -DNDEBUG): all four registered tests pass. The new test restores assertions and checks 74 public-API cases, including sorted/reverse/shuffled names, the 64/65 transition, growth boundaries, exact reference identities and indexed error/reset paths. The same 74 cases pass against the original library.
  • Scoped ASan/UBSan/LSan checks pass with leak detection enabled. The existing reader-test leak requires tests/test-reader: Fix memory leak #316's test-only cleanup in both sanitizer arms; that cleanup is not included here. Index-only failure injection checks every growth point through 16,000 anchors, not every library allocation. The largest requested index block there is 256 KiB; documents with at most 64 anchors allocate no index.
  • Two complete, order-balanced load-cycle observations with four paired process blocks per case (-O2 libraries, 14 cases, 112 fresh processes per observation). At 16,000 anchors, the median B/C ratios are about 42x without aliases and 83x with repeated references to the last anchor. These stress linear lookup and are not application-throughput claims. Small cases have tradeoffs: first-run empty/ordinary/small medians range from 0.963x to 1.038x, including about 3.7% slowdown at eight references. Four pairs on one shared host give limited precision.

Portable reproducer, both complete observations and limitations are on a separate fork branch, outside this PR's four-file diff. The portable runner itself was executed successfully against both exact native archives.

Autotools registration is included, but local make test-all, the external YAML test-suite corpus, other platforms and hosted CI remain unverified. Development was AI-assisted.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Quadratic CPU in yaml_parser_load: linear anchor/alias lookup (O(n²))

1 participant