perf: drop node positions after metadata by ovflowd · Pull Request #1236 · nodejs/doc-kit

ovflowd · 2026-11-03T19:33:43Z

Description

This PR drops position from every node of metadata's entries once they're parsed, with one walk over each entry's content. Nothing after metadata reads where a node came from in the source: the type warnings that print positions run while it parses. On Node.js's docs that's ~480k objects, about 80MB, that the main thread no longer holds or copies to and from the workers. Builds are 2-5% faster and peak 2-5% lower at every thread count (table below).

json-simple writes the metadata as it is, so it no longer has them either, and is about a third of its size.

Why it's built this way

Why drop them at the end of metadata, and not earlier? One thing still needs them: the invalid type annotation warning in resolveTypes.mjs. It passes node.position to the logger, and both the console and GitHub transports print that line. It runs early in parseApiDoc, and nothing after it reads positions. So the walk goes at the end, where the entries exist, and touches only the nodes they keep. They can't be skipped at parse time either, because mdast-util-from-markdown sets position on every node it creates and has no option to turn that off.

Does anything downstream lose information? Only json-simple's output. In packages/*/src, the logger transports are the only code that reads a node's position, and only that warning feeds them. callIfBefore in utils/unist.mjs compares positions, but only its tests call it. yamlPosition is declared in metadata/types.d.ts but never set. Nothing in node-legacy or react reads positions. As a side effect, the hast trees built from these entries get smaller too: mdast-util-to-hast copies a position only when the source node has one.

Why one walk over content? Every entry's subtree contains its heading. visitStability finds the stability blockquote in that same subtree and changes it in place without removing it. So one visit(content) per entry covers all three, with no loop over content, heading and stability and no check for a missing stability.

Why delete and not position = undefined? A key set to undefined stays on the node, and structuredClone still copies it to and from the workers. That would save neither the property nor the copy.

The CI comment says web got 48% slower. That's a single run on a shared runner. The same comment has api-links 21% slower, and api-links builds from ast-js, so it never runs parseApiDoc. Across interleaved runs on one machine (below), both builds are 2-5% faster at every thread count.

Does this still matter once the streaming stack lands? Yes, and more. It doesn't touch the stack, but entries still get copied to and from workers there, just in chunks. At 4 threads the stack alone peaks 34% above its base; with this PR and #1235 on top, 3%.

Benchmarks

Full Node.js docs on Node 26. Each number is the median of 5 interleaved runs (3 at one thread). Peak is the RSS summed over the whole process tree. Site is -t orama-db -t json -t llms-txt -t section-pages, legacy is -t legacy-html-all -t legacy-json-all. The baseline is 61fa0b9, the commit this branch sits on.

Build	Threads	`61fa0b9`	This PR	Time	Memory
site	1	22.80s, 1747MB	22.15s, 1690MB	-3%	-3%
site	4	9.25s, 1797MB	8.75s, 1762MB	-5%	-2%
site	6	8.50s, 2216MB	8.04s, 2110MB	-5%	-5%
legacy	1	9.26s, 1260MB	9.10s, 1227MB	-2%	-3%
legacy	4	4.24s, 1539MB	4.04s, 1502MB	-5%	-2%
legacy	6	4.20s, 1972MB	3.99s, 1920MB	-5%	-3%

Validation

Added drops position from every node of an entry, which checks that no node of a parsed entry's content has a position, and that its heading and stability don't either. json-simple is the only target whose output changes, and the <api>.json files json and legacy-json both write already race on main.

Related Issues

Split out of #1205.

Check List

I have read the Contributing Guidelines and made commit messages that follow the guideline.
I have run node --run test and all tests passed.
I have check code formatting with node --run format:check & node --run lint.
I've covered new added functionality with unit tests if necessary.

vercel · 2026-11-03T19:33:49Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
api-docs-tooling	Ready	Preview	Nov 11, 2026 12:07am UTC

codecov · 2026-11-03T19:35:10Z

Codecov Report

❌ Patch coverage is 92.68293% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 94.18%. Comparing base (61fa0b9) to head (2ab0ad0).
⚠️ Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
...generators/metadata/utils/__tests__/parse.test.mjs	90.62%	3 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1236      +/-   ##
==========================================
+ Coverage   94.13%   94.18%   +0.04%     
==========================================
  Files         291      291              
  Lines       28900    29255     +355     
  Branches     2932     2976      +44     
==========================================
+ Hits        27206    27553     +347     
- Misses       1672     1680       +8     
  Partials       22       22

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

github-actions · 2026-11-03T19:35:55Z