fix(rehype-shiki): only import every grammar without a highlighter by ovflowd · Pull Request #9212 · nodejs/nodejs.org

ovflowd · 2026-10-10T16:24:31Z

Description

This PR makes @node-core/rehype-shiki's plugin import index.mjs only when it has to create a highlighter itself.

plugin.mjs imported index.mjs statically, and that module imports every grammar Shiki bundles (LANGS) as soon as it's loaded. So a caller passing its own highlighter still paid for all of them: importing the plugin loaded 260 grammar modules on every thread that used it. It's now a dynamic import, inside the branch that creates the default highlighter, and a patch changeset releases it.

Validation

The package's unit tests (10) pass, and so do lint:js and Prettier.
Importing plugin.mjs loads 260 grammar modules before this change and none after it, counted with a module load hook.

Related Issues

Refs: nodejs/doc-kit#1156, which highlights with its own highlighter that loads grammars on demand, and can then use this plugin instead of a copy of it.

Check List

I have read the Contributing Guidelines and made commit messages that follow the guideline.
I have run pnpm format to ensure the code follows the style guide.
I have run pnpm test to check if all tests are passing.
I have run pnpm build to check if the website builds without errors.
I've covered new added functionality with unit tests if necessary.

vercel · 2026-10-10T16:24:37Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
nodejs-org	Ready	Preview	Oct 10, 2026 6:01pm UTC

codecov · 2026-10-10T16:26:11Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 86.15%. Comparing base (bbbdde7) to head (e1c50ce).
⚠️ Report is 1 commits behind head on main.
✅ All tests successful. No failed tests found.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #9212      +/-   ##
==========================================
+ Coverage   86.07%   86.15%   +0.08%     
==========================================
  Files          86       86              
  Lines        6060     6075      +15     
  Branches      359      360       +1     
==========================================
+ Hits         5216     5234      +18     
+ Misses        840      837       -3     
  Partials        4        4

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

Copilot

🟡 Changes recommended

The supplied-highlighter path needs an automated regression test proving bundled grammars remain unloaded.

1 open finding

Add test for custom highlighter import avoidance · New

What changed in this PR

Defers loading bundled Shiki grammars when callers provide their own highlighter, reducing import overhead for consumers such as nodejs/doc-kit.

Changes:

Dynamically imports the default highlighter only when needed.
Adds a patch changeset.

File	Description
`packages/rehype-shiki/src/plugin.mjs`	Lazily loads the grammar-heavy default highlighter.
`.changeset/lazy-rehype-shiki-highlighter.md`	Records the patch release.

🧠 Review effort: Balanced

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

The Shiki plugin registered every bundled language (~250 grammars) in each thread that highlighted code, which cost ~2s and ~100MB per thread and made every highlight several times slower, as each code block was matched against grammars it never uses. Importing it also imported all of them, through `@node-core/rehype-shiki`'s `LANGS` and its plugin, on every thread loading `jsx-ast`, the main thread included. The highlighter now registers a bundled language the first time code in it is highlighted, along with the bundled languages a configured one embeds, and lists the bundled ones from their metadata alone, without importing `LANGS`. The themes are given to Shiki by name, which it keeps parsed instead of parsing them for every highlight. Importing `@node-core/rehype-shiki`'s plugin still imports every grammar until nodejs/nodejs.org#9212 is released. The grammars now come from doc-kit's own `shiki` dependency (4.4.3) rather than the copy `@node-core/rehype-shiki` pins (4.3.1). Its C++ grammar highlights types and template arguments differently, which shows on the Node.js docs' C++ examples. Assisted-by: Claude Opus 5.5 <noreply@anthropic.com>

github-actions · 2026-10-10T16:35:55Z

📦 Build Size Comparison

Summary

Metric	Value
Old Total First Load JS	7.20 MB
New Total First Load JS	7.20 MB
Delta	0 B (0.00%)

The Shiki plugin registered every bundled language (~250 grammars) in each thread that highlighted code, which cost ~2s and ~100MB per thread and made every highlight several times slower, as each code block was matched against grammars it never uses. Importing it also imported all of them, through `@node-core/rehype-shiki`'s `LANGS` and its plugin, on every thread loading `jsx-ast`, the main thread included. The highlighter now registers a bundled language the first time code in it is highlighted, along with the bundled languages a configured one embeds, and lists the bundled ones from their metadata alone, without importing `LANGS`. The themes are given to Shiki by name, which it keeps parsed instead of parsing them for every highlight. Importing `@node-core/rehype-shiki`'s plugin still imports every grammar until nodejs/nodejs.org#9212 is released. The grammars now come from doc-kit's own `shiki` dependency (4.4.3) rather than the copy `@node-core/rehype-shiki` pins (4.3.1). Its C++ grammar highlights types and template arguments differently, which shows on the Node.js docs' C++ examples. Assisted-by: Claude Opus 5.5 <noreply@anthropic.com>

The plugin imported `index.mjs` statically, and that module imports every grammar Shiki bundles (`LANGS`) as soon as it's loaded, so a caller passing its own highlighter still paid for all of them: 260 grammar modules on each thread that loads the plugin. It's now imported only when the plugin has to create a highlighter itself. This lets doc-kit, which highlights with a highlighter that loads grammars on demand, use the plugin rather than a copy of it. Assisted-by: Claude Opus 5.5 <noreply@anthropic.com>

ovflowd · 2026-10-10T21:00:49Z

@nodejs/nodejs-website requesting fast-track please 👍 or 👎

The Shiki plugin registered every bundled language (~250 grammars) in each thread that highlighted code, which cost ~2s and ~100MB per thread and made every highlight several times slower, as each code block was matched against grammars it never uses. Importing it also imported all of them, through `@node-core/rehype-shiki`'s `LANGS` and its plugin, on every thread loading `jsx-ast`, the main thread included. The highlighter now registers a bundled language the first time code in it is highlighted, along with the bundled languages a configured one embeds, and lists the bundled ones from their metadata alone, without importing `LANGS`. The themes are given to Shiki by name, which it keeps parsed instead of parsing them for every highlight. `@node-core/rehype-shiki` is updated to 1.5.1, whose plugin only imports every grammar when it isn't given a highlighter (nodejs/nodejs.org#9212). The grammars now come from doc-kit's own `shiki` dependency (4.4.3), which 1.5.1 shares, rather than the copy 1.5.0 pinned (4.3.1). Its C++ grammar highlights types and template arguments differently, which shows on the Node.js docs' C++ examples. Assisted-by: Claude Opus 5.5 <noreply@anthropic.com>

The Shiki plugin registered every bundled language (~250 grammars) in each thread that highlighted code, which cost ~2s and ~100MB per thread and made every highlight several times slower, as each code block was matched against grammars it never uses. Importing it also imported all of them, through `@node-core/rehype-shiki`'s `LANGS` and its plugin, on every thread loading `jsx-ast`, the main thread included. The highlighter now registers a bundled language the first time code in it is highlighted, along with the bundled languages a configured one embeds, and lists the bundled ones from their metadata alone, without importing `LANGS`. The themes are given to Shiki by name, which it keeps parsed instead of parsing them for every highlight, named after the configured themes, as Shiki loads a theme given for both schemes once. A bundled language is registered with the bundled languages it embeds, lazily too (a fenced code block in Markdown): Shiki only highlights those once they are registered, so registering them later would make code highlight differently depending on what was highlighted before it. For the same reason, a bundled grammar injecting into others (`ts-tags`, into JavaScript's tagged templates) only injects into the language it was imported for. `@node-core/rehype-shiki` is updated to 1.5.1, whose plugin only imports every grammar when it isn't given a highlighter (nodejs/nodejs.org#9212). The grammars now come from doc-kit's own `shiki` dependency (4.4.3), which 1.5.1 shares, rather than the copy 1.5.0 pinned (4.3.1). Its C++ grammar highlights types and template arguments differently, which shows on the Node.js docs' C++ examples. Assisted-by: Claude Opus 5.5 <noreply@anthropic.com>

The Shiki plugin registered every bundled language (~250 grammars) in each thread that highlighted code, which cost ~2s and ~100MB per thread and made every highlight several times slower, as each code block was matched against grammars it never uses. Importing it also imported all of them, through `@node-core/rehype-shiki`'s `LANGS` and its plugin, on every thread loading `jsx-ast`, the main thread included. The highlighter now registers a bundled language the first time code in it is highlighted, along with the bundled languages a configured one embeds, and lists the bundled ones from their metadata alone, without importing `LANGS`. The themes are given to Shiki by name, which it keeps parsed instead of parsing them for every highlight, named after the configured themes, as Shiki loads a theme given for both schemes once. A bundled language is registered with the bundled languages it embeds, lazily too (a fenced code block in Markdown): Shiki only highlights those once they are registered, so registering them later would make code highlight differently depending on what was highlighted before it. For the same reason, a bundled grammar injecting into others (`ts-tags`, into JavaScript's tagged templates) only injects into the language it was imported for. `@node-core/rehype-shiki` is updated to 1.5.1, whose plugin only imports every grammar when it isn't given a highlighter (nodejs/nodejs.org#9212). The grammars now come from doc-kit's own `shiki` dependency (4.4.3), which 1.5.1 shares, rather than the copy 1.5.0 pinned (4.3.1). Its C++ grammar highlights types and template arguments differently, which shows on the Node.js docs' C++ examples. Assisted-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Aviv Keller <me@aviv.sh>

* perf(core): load Shiki grammars on demand The Shiki plugin registered every bundled language (~250 grammars) in each thread that highlighted code, which cost ~2s and ~100MB per thread and made every highlight several times slower, as each code block was matched against grammars it never uses. Importing it also imported all of them, through `@node-core/rehype-shiki`'s `LANGS` and its plugin, on every thread loading `jsx-ast`, the main thread included. The highlighter now registers a bundled language the first time code in it is highlighted, along with the bundled languages a configured one embeds, and lists the bundled ones from their metadata alone, without importing `LANGS`. The themes are given to Shiki by name, which it keeps parsed instead of parsing them for every highlight, named after the configured themes, as Shiki loads a theme given for both schemes once. A bundled language is registered with the bundled languages it embeds, lazily too (a fenced code block in Markdown): Shiki only highlights those once they are registered, so registering them later would make code highlight differently depending on what was highlighted before it. For the same reason, a bundled grammar injecting into others (`ts-tags`, into JavaScript's tagged templates) only injects into the language it was imported for. `@node-core/rehype-shiki` is updated to 1.5.1, whose plugin only imports every grammar when it isn't given a highlighter (nodejs/nodejs.org#9212). The grammars now come from doc-kit's own `shiki` dependency (4.4.3), which 1.5.1 shares, rather than the copy 1.5.0 pinned (4.3.1). Its C++ grammar highlights types and template arguments differently, which shows on the Node.js docs' C++ examples. Assisted-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Aviv Keller <me@aviv.sh> * perf(react): embed highlighted code as static markup Every highlighted token of every code block, signature and type became a hast element, then a JSX element, then generated code, only to be rendered back into the same markup: most of what `jsx-ast` allocated, and much of what the pages' code took to compile and render. Highlighted code is static, as islands adopt it without rendering it again, so it now reaches the pages as HTML, written by `hast-util-to-html` and held by a `<code>` through `dangerouslySetInnerHTML`. Code blocks are embedded by a plugin running after Shiki. Types and signatures are embedded as they are highlighted, before `rehype-raw`, which would otherwise parse each of their tokens again. It is the HTML Preact rendered, spelled differently in places: `hast-util-to-html` writes `<` as `<`, leaves `"` unescaped in text, and keeps Shiki's token styles without a trailing `;`. Assisted-by: Claude Opus 5.5 <noreply@anthropic.com> * perf(react): render `all.html` in the worker pool The pool renders a single task on the calling thread, so `all.html`, rendered on its own after the other pages, was rendered on the main thread. That held the whole site in its heap, and kept the pool from shutting its idle workers down until it was done. It is now rendered alongside the other pages, first, as it takes by far the longest. Assisted-by: Claude Opus 5.5 <noreply@anthropic.com> * perf: keep worker-only code off the main thread The main thread loads every generator of a run, but only ever calls `generate`; building the pages is the workers' job. Still, the generators imported what only their workers use, so the main thread loaded the TypeScript parser (WASM) and what building a page takes. Those are now imported where they are used. `getFullName` also moves, next to `getClassNames` in `utils/hast.mjs`: `buildBarProps`, which `section-pages` imports, and `buildContent` only need a page's full name, not the code that builds and highlights signatures. Assisted-by: Claude Opus 5.5 <noreply@anthropic.com> * perf(react): keep the `#theme/config` source off the rendering threads The threads rendering pages load the `html` generator's module, and so everything `generate` imports, along with `processing.mjs`. That included `config.mjs`, which loads Shiki for the languages' display names and a Markdown processor of its own: ~14MB and ~75ms per worker, for what only the main thread uses, while the workers render pages, which is when the build peaks. `createVirtualImports` moves into `config.mjs`, which `generate` imports when it bundles the site, so the main thread alone loads it. Assisted-by: Claude Opus 5.5 <noreply@anthropic.com> * docs(core): describe the thread ceiling without Shiki's grammars A worker that highlights code no longer registers every grammar Shiki bundles, only those of the code it highlighted. The comment now says what holds for every generator: each worker has a heap of its own, with the libraries and the pages it is working on. Assisted-by: Claude Opus 5.5 <noreply@anthropic.com> * perf(core): limit each worker's heap to 512MB V8 lets a heap grow to several times its live data before collecting it, the more the higher its limit: four times from 2GB up, and on a machine with plenty of memory every worker's limit is 4GB, so each held several times what it was using. Each worker's old space is now limited to `WORKER_HEAP_SIZE` (512MB). The biggest page of the Node.js docs, `all.html`, takes ~300MB. Node's `--max-old-space-size` replaces the limit, to raise it, and it's what limits the main thread, which runs the generators with one thread. On Node core's build with 4 threads, the peak goes from 2.02GB to 1.67GB on Node 26, and from 2.44GB to 2.19GB on Node 24, for 3-5% more time spent collecting garbage. The build passes with workers limited to as little as ~350MB. Assisted-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Aviv Keller <me@aviv.sh> * perf(core): end idle workers after 250ms instead of a second A worker only ended a second after its last task, so the `jsx-ast` workers and their heaps (~800MB on the Node.js docs) were still there while the main thread bundled the site, which is when the build peaked. They now end a quarter of a second after running out of work, soon enough to be gone while the site is bundled. On Node core's build with 4 threads on Node 26, the peak goes from ~2.31GB to ~1.91GB. Workers also end in some of the gaps between generators, so the build starts 10 or 11 workers rather than 8, which costs ~0.3s. Assisted-by: Claude Opus 5.5 <noreply@anthropic.com> * perf(react): highlight each type once per thread Of the ~20,000 types the Node.js docs highlight, 741 are different: `{string}` alone is on most pages. Each thread now keeps the markup of the types it highlighted, by highlighter, so a type it has seen is only embedded again, saving ~4% of the build's time and ~5% of its CPU. Assisted-by: Claude Opus 5.5 <noreply@anthropic.com> * perf(core): write GitHub Actions commands without `@actions/core` Every thread loads the logger, and with it each transport: the GitHub one imported `@actions/core`, and through it `undici`, for the four functions printing workflow commands. That was 237 modules, 19MB and ~80ms on every thread, whichever transport is used. The transport now writes the commands itself, escaped as `@actions/core` does, for the same output: 10 modules, 3MB and 7ms. Assisted-by: Claude Opus 5.5 <noreply@anthropic.com> * perf(react): let go of the bundler before the pages render `generate` bundled the site and compiled the pages' programs in its own scope, so everything it set up for that, the bundler, the program builder and the compile function, stayed reachable while the pages rendered. With one thread, the main thread renders them all, and held it all the while. The bundling and the compiling move to `buildPrograms`, which returns the pages' tasks for the pool, so all of it is let go once it returns. On Node core's build with one thread on Node 26, the peak goes from ~2.66GB to ~2.45GB (medians of 8 runs). With four threads, where the workers render the pages, it doesn't change. Assisted-by: Claude Opus 5.5 <noreply@anthropic.com> * chore(legacy-html): mark its Shiki plugin as deprecated It will be removed soon. Assisted-by: Claude Opus 5.5 <noreply@anthropic.com> * perf: don't minify by default The HTML generators minified every page with SWC's HTML minifier, compiled to WASM. Its memory grows to about twelve times the biggest page it is given, and is never returned: each thread rendering pages kept it until it ended, and on Node.js 24, the worker that minified the Node.js docs' 33MB `all.html` collected garbage on nearly every allocation from then on, as V8 found that memory too large for its heap limit. Minifying also took a fifth of the build's CPU. For that, the Node.js docs' pages were 5.9% smaller, but only 3.2% once gzipped and 2.1% with brotli, which is how hosts serve them: Cloudflare, Vercel, Netlify and GitHub Pages compress by default. So `minify` now defaults to `false`, the minifier is only loaded when it is enabled, and the publishing docs explain how to serve the output compressed. The `html` generator's scripts and styles stay minified, unless `vite.build.minify` says otherwise. On Node core's build with 4 threads on Node.js 26, the build takes 7.7s instead of 10.5s and peaks at ~1.68GB instead of ~2.12GB; with one thread, 15.2s instead of 22.5s. On Node.js 24, 7.9s instead of 51.7s. Assisted-by: Claude Opus 5.5 <noreply@anthropic.com> * chore: add a changeset for the build memory work Assisted-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Aviv Keller <me@aviv.sh> --------- Co-authored-by: Aviv Keller <me@aviv.sh>

ovflowd requested a review from a team as a code owner October 10, 2026 16:24

Copilot AI balanced review requested due to automatic review settings October 10, 2026 16:24

vercel Bot deployed to Preview October 10, 2026 16:24 View deployment

Copilot started reviewing on behalf of ovflowd October 10, 2026 16:25 View session

Copilot AI reviewed Oct 10, 2026

View reviewed changes

Comment thread packages/rehype-shiki/src/plugin.mjs

ovflowd mentioned this pull request Oct 10, 2026

perf: reduce build memory and time nodejs/doc-kit#1156

Merged

4 tasks done

avivkeller approved these changes Oct 10, 2026

View reviewed changes

ovflowd force-pushed the fix/rehype-shiki-lazy-highlighter branch from 36ac193 to e1c50ce Compare October 10, 2026 18:00

vercel Bot deployed to Preview October 10, 2026 18:01 View deployment

ovflowd added the fast-track Fast Tracking PRs label Oct 10, 2026

AugustinMauroy approved these changes Oct 11, 2026

View reviewed changes

ovflowd added this pull request to the merge queue Oct 11, 2026

Merged via the queue into main with commit 922aae3 Oct 11, 2026
22 checks passed

ovflowd deleted the fix/rehype-shiki-lazy-highlighter branch October 11, 2026 12:37

github-actions Bot mentioned this pull request Oct 11, 2026

chore: version packages #9213

Merged

Navigation Menu

Repository navigation

fix(rehype-shiki): only import every grammar without a highlighter - #9212

fix(rehype-shiki): only import every grammar without a highlighter#9212
ovflowd merged 1 commit into
mainfrom
fix/rehype-shiki-lazy-highlighter

ovflowd commented Oct 10, 2026

Uh oh!

vercel Bot commented Oct 10, 2026 •

edited

Loading

Uh oh!

codecov Bot commented Oct 10, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

github-actions Bot commented Oct 10, 2026 •

edited

Loading

Uh oh!

ovflowd commented Oct 10, 2026

Uh oh!

Uh oh!

This branch was successfully deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

ovflowd commented Oct 10, 2026

Description

Validation

Related Issues

Check List

Uh oh!

vercel Bot commented Oct 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov Bot commented Oct 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

Uh oh!

github-actions Bot commented Oct 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

📦 Build Size Comparison

Summary

Uh oh!

ovflowd commented Oct 10, 2026

Uh oh!

Uh oh!

This branch was successfully deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

vercel Bot commented Oct 10, 2026 •

edited

Loading

codecov Bot commented Oct 10, 2026 •

edited

Loading

github-actions Bot commented Oct 10, 2026 •

edited

Loading