perf: highlight with fewer grammars by default by ovflowd · Pull Request #1235 · nodejs/doc-kit

ovflowd · 2026-11-03T19:33:40Z

Description

This PR highlights code with fewer grammars by default, as each one takes its own memory on every thread, and time to register. JavaScript (js, mjs, cjs, javascript) now uses TypeScript's grammar, which already highlights types, and C++ (cpp, c++) uses C's. JavaScript's grammar alone is ~60MB on every thread: on Node.js's docs the site build peaks 95MB lower at 4 threads and 207MB lower at 6, and builds take 1-6% less time.

In the light theme JavaScript comes out the same, except one block of console output marked as js. Nord, the dark theme, loses a few colors it only gives JavaScript's scopes, and C++-only keywords lose theirs. langAlias takes precedence, so { cpp: 'cpp', 'c++': 'cpp' } brings C++'s own grammar back.

doc-kit now resolves aliases itself, since Shiki replaced configured ones with a grammar's own as it registered. Aliases also follow other aliases and match in any case. resolveLanguage returns, and transformers see, the language code is highlighted as (typescript for mjs).

Why it's built this way

What's wrong with JavaScript's grammar? Nothing, we just pay for it twice. Each worker has its own Shiki instance and Oniguruma engine, so every grammar a thread registers costs that thread its own memory. Grammars are already registered lazily, but Node.js's docs have 2,632 js/mjs/cjs blocks spread over nearly every page, so every thread ended up with JavaScript's grammar next to TypeScript's, which types are highlighted with anyway. There are only 5 ts blocks.

Doesn't JavaScript's grammar still load? On some threads, yes. Embedded grammars load with the grammar that embeds them: http (http.md, http2.md) embeds JavaScript through graphql, and coffee (module.md) embeds it directly. So the saving is on every thread that doesn't render those three pages.

And C++? The docs have 29 cpp blocks and 211 c ones. C++'s grammar brings cpp-macro, glsl and regexp with it, ~850KB of grammar source against 78KB for C, which is loaded anyway for the c blocks.

Why resolve aliases in doc-kit and not with Shiki's langAlias? When a grammar registers, Shiki overwrites the alias map with that grammar's own aliases (this._alias[alias] = lang.name). With js: 'typescript' set in Shiki, the first http or coffee block would register JavaScript's grammar and flip js back to it in the middle of a build, so a page would highlight differently depending on what its thread had highlighted before. resolveLanguage returns typescript because Shiki only knows TypeScript as ts, cts and mts, so callers have to highlight with the name it returns.

Why does a configured alias turn the defaults off? So getting a grammar back doesn't need a new option. With { cpp: 'cpp' }, a cpp block stops at the repeat and gets C++'s grammar. The defaults only apply when no configured alias matched.

Why lowercase? main already lowercased the language before Shiki's lookup, and Shiki keeps its bundled names and aliases in lowercase. Configured keys are lowercased now too, so { MyConf: 'ini' } matches myconf as well.

Why does one thread use more memory on the site build? I don't know yet. On one thread, http.md and module.md register JavaScript's grammar anyway, so only C++'s grammars drop out and there's little to save. It went from 1747 to 1838MB (+5%); this PR ran 1834 to 1850MB and base ran 1742, 1747 and 1982MB, so with 3 runs it may well be real. The legacy build on one thread went down 87MB, but base was noisy there too (1136 to 1270MB).

Does it matter with the stack? The saving grows with threads, and #1243 raises the default to up to 6. The stack alone is +34% memory on the site build at 4 threads; with this PR and #1236 it's -20% / -22% time and +3% / -1% memory at 4 / 6 threads, against main before #1237 (61fa0b9). I haven't measured this PR alone on top of the stack.

Benchmarks

Node.js's docs on Node.js 26. This PR against its base (61fa0b9), medians of 5 interleaved runs (3 on one thread). Peak memory is RSS summed over the process tree. The site build is -t orama-db -t json -t llms-txt -t section-pages and the legacy build is -t legacy-html-all -t legacy-json-all.

Build	Threads	Time	Peak memory
site	1	22.80s → 22.43s (-2%)	1747 → 1838MB (+5%)
site	4	9.25s → 8.99s (-3%)	1797 → 1702MB (-5%)
site	6	8.50s → 8.35s (-2%)	2216 → 2009MB (-9%)
legacy	1	9.26s → 8.91s (-4%)	1260 → 1173MB (-7%)
legacy	4	4.24s → 4.00s (-6%)	1539 → 1646MB (+7%)
legacy	6	4.20s → 4.17s (-1%)	1972 → 1873MB (-5%)

The legacy build at 4 threads went up, and it doesn't look like noise: 4 of 5 runs (1637 to 1674MB) were above every base run (1526 to 1603MB). I haven't found why yet. At 6 threads the same build is 5% lower.

Validation

Added highlights JavaScript as TypeScript, and C++ as C, takes configured aliases over the default ones, follows configured aliases, and finds languages named in any case and highlights with a language of `langs` an alias leads to. node --run test, lint and format:check are clean.

The memory and time are from the benchmarks above.

Related Issues

Split out of #1205.

Check List

I have read the Contributing Guidelines and made commit messages that follow the guideline.
I have run node --run test and all tests passed.
I have check code formatting with node --run format:check & node --run lint.
I've covered new added functionality with unit tests if necessary.

vercel · 2026-11-03T19:33:42Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
api-docs-tooling	Ready	Preview	Nov 11, 2026 12:07am UTC

codecov · 2026-11-03T19:34:57Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 94.23%. Comparing base (61fa0b9) to head (b570722).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1235      +/-   ##
==========================================
+ Coverage   94.13%   94.23%   +0.09%     
==========================================
  Files         291      292       +1     
  Lines       28900    29331     +431     
  Branches     2932     2987      +55     
==========================================
+ Hits        27206    27640     +434     
+ Misses       1672     1669       -3     
  Partials       22       22

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

cloudflare-workers-and-pages · 2026-11-03T19:38:48Z

🚀 Deploying Preview to Cloudflare 🚀