Skip to content

refactor(core): load generators by import specifier - #958

Open
avivkeller wants to merge 1 commit into
mainfrom
refactor/specifier-loading
Open

refactor(core): load generators by import specifier#958
avivkeller wants to merge 1 commit into
mainfrom
refactor/specifier-loading

Conversation

@avivkeller

@avivkeller avivkeller commented Jul 30, 2026

Copy link
Copy Markdown
Member

Generators are no longer resolved through a static registry.

Now, they're now loaded dynamically by import specifier. --target accepts either a built-in shorthand (web, legacy-html, …) or any import specifier that resolves to a generator module (e.g. some-package/generator, ./my-generator.mjs). A generator's dependsOn field is likewise now a full import specifier.

Copilot AI review requested due to automatic review settings July 30, 2026 21:30
@vercel

vercel Bot commented Jul 30, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
api-docs-tooling Ready Ready Preview Jul 30, 2026 9:31pm

Request Review

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@codecov

codecov Bot commented Jul 30, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 94.12916% with 30 lines in your changes missing coverage. Please review.
✅ Project coverage is 88.82%. Comparing base (70b7526) to head (c8553db).

Files with missing lines Patch % Lines
packages/core/src/generators/loader.mjs 75.00% 23 Missing and 1 partial ⚠️
packages/core/bin/commands/generate.mjs 0.00% 5 Missing ⚠️
packages/core/src/generators.mjs 97.22% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #958      +/-   ##
==========================================
+ Coverage   86.50%   88.82%   +2.31%     
==========================================
  Files         195      197       +2     
  Lines       17964    18132     +168     
  Branches     1632     1664      +32     
==========================================
+ Hits        15540    16105     +565     
+ Misses       2418     2020     -398     
- Partials        6        7       +1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@avivkeller
avivkeller marked this pull request as ready for review July 30, 2026 21:32
@avivkeller
avivkeller requested a review from a team as a code owner July 30, 2026 21:32
@cursor

cursor Bot commented Jul 30, 2026

Copy link
Copy Markdown

PR Summary

Medium Risk
Touches the core generator orchestration, worker loading, and configuration merge path; regressions could break any doc-kit generate run, though behavior is largely preserved behind aliases and added tests.

Overview
Generators are loaded at runtime by import specifier instead of a static in-process registry. The pipeline, cache, config defaults, and worker threads all key off resolved specifiers (with shorthand names mapped to @node-core/doc-kit/... paths).

--target / target now accepts built-in shorthands (web, legacy-html, …), package subpath specifiers, or local file paths (converted to file: URLs). dependsOn is a full import specifier, not a registry key. Built-in generators are exposed via new package.json exports entries; generators/index.mjs is only an alias table and no longer imports generator implementations.

Generator modules drop createLazyGenerator in favor of a default export object that wires generate / processChunk directly. Parallel workers pass generatorSpecifier into chunk-worker.mjs, which import()s the module in each worker. Configuration resolution loads the target dependency closure before merging generator defaults. Docs and the changeset describe the new extension model for third-party generators.

Signature parsing helpers move from legacy-json to utils/signature (shared by jsx-ast / legacy-json). Tests and fixtures are updated to mock or call loader.mjs and to pass target where config loading requires it.

Reviewed by Cursor Bugbot for commit c8553db. Bugbot is set up for automated code reviews on this repo. Configure here.

@github-actions

Copy link
Copy Markdown
Contributor

api-links Generator

Output: 1 file differs

apilinks.json
Expected values to be strictly deep-equal:
+ actual - expected
... Skipped lines

  {
    'Agent.defaultMaxSockets': 'https://github.com/nodejs/node/blob/HEAD/lib/_http_agent.js#L291',
    'Buffer.alloc': 'https://github.com/nodejs/node/blob/HEAD/lib/buffer.js#L435',
    'Buffer.allocUnsafe': 'https://github.com/nodejs/node/blob/HEAD/lib/buffer.js#L449',
    'Buffer.allocUnsafeSlow': 'https://github.com/nodejs/node/blob/HEAD/lib/buffer.js#L461',
...
    'agent.addRequest': 'https://github.com/nodejs/node/blob/HEAD/lib/_http_agent.js#L356',
+   'agent.createConnection': 'https://github.com/nodejs/node/blob/HEAD/lib/_http_agent.js#L295',
-   'agent.createConnection': 'https://github.com/nodejs/node/blob/HEAD/lib/https.js#L332',
    'agent.createSocket': 'https://github.com/nodejs/node/blob/HEAD/lib/_http_agent.js#L427',
    'agent.destroy': 'https://github.com/nodejs/node/blob/HEAD/lib/_http_agent.js#L660',
+   'agent.getName': 'https://github.com/nodejs/node/blob/HEAD/lib/_http_agent.js#L325',
+   'agent.keepSocketAlive': 'https://github.com/nodejs/node/blob/HEAD/lib/_http_agent.js#L616',
-   'agent.getName': 'https://github.com/nodejs/node/blob/HEAD/lib/https.js#L515',
-   'agent.keepSocketAlive': 'https://github.com/nodejs/node/blob/HEAD/lib/https.js#L484',
    'agent.removeSocket': 'https://github.com/nodejs/node/blob/HEAD/lib/_http_agent.js#L555',
    'agent.reuseSocket': 'https://github.com/nodejs/node/blob/HEAD/lib/_http_agent.js#L652',
    'assert.assert': 'https://github.com/nodejs/node/blob/HEAD/lib/assert.js#L185',
    'asyncResource.asyncId': 'https://github.com/nodejs/node/blob/HEAD/lib/async_hooks.js#L243',
    'asyncResource.bind': 'https://github.com/nodejs/node/blob/HEAD/lib/async_hooks.js#L275',
...
    'server.address': 'https://github.com/nodejs/node/blob/HEAD/lib/net.js#L2680',
+   'server.close': 'https://github.com/nodejs/node/blob/HEAD/lib/_http_server.js#L674',
+   'server.closeAllConnections': 'https://github.com/nodejs/node/blob/HEAD/lib/_http_server.js#L684',
+   'server.closeIdleConnections': 'https://github.com/nodejs/node/blob/HEAD/lib/_http_server.js#L696',
-   'server.close': 'https://github.com/nodejs/node/blob/HEAD/lib/net.js#L2813',
-   'server.closeAllConnections': 'https://github.com/nodejs/node/blob/HEAD/lib/https.js#L124',
-   'server.closeIdleConnections': 'https://github.com/nodejs/node/blob/HEAD/lib/https.js#L126',
    'server.getConnections': 'https://github.com/nodejs/node/blob/HEAD/lib/net.js#L2775',
    'server.listen': 'https://github.com/nodejs/node/blob/HEAD/lib/net.js#L2477',
    'server.ref': 'https://github.com/nodejs/node/blob/HEAD/lib/net.js#L2926',
+   'server.setTimeout': 'https://github.com/nodejs/node/blob/HEAD/lib/_http_server.js#L712',
-   'server.setTimeout': 'https://github.com/nodejs/node/blob/HEAD/lib/https.js#L128',
    'server.unref': 'https://github.com/nodejs/node/blob/HEAD/lib/net.js#L2935',
+   'server[SymbolAsyncDispose]': 'https://github.com/nodejs/node/blob/HEAD/lib/_http_server.js#L680',
-   'server[SymbolAsyncDispose]': 'https://github.com/nodejs/node/blob/HEAD/lib/net.js#L2853',
    'server[SymbolAsyncIterator]': 'https://github.com/nodejs/node/blob/HEAD/lib/net.js#L2860',
    'server[kDeserialize]': 'https://github.com/nodejs/node/blob/HEAD/lib/net.js#L2398',
    'server[kTransferList]': 'https://github.com/nodejs/node/blob/HEAD/lib/net.js#L2371',
    'server[kTransfer]': 'https://github.com/nodejs/node/blob/HEAD/lib/net.js#L2376',
+   'server[undefined]': 'https://github.com/nodejs/node/blob/HEAD/lib/_http_server.js#L719',
-   'server[undefined]': 'https://github.com/nodejs/node/blob/HEAD/lib/net.js#L2890',
    'serverresponse._finish': 'https://github.com/nodejs/node/blob/HEAD/lib/_http_server.js#L249',
    'serverresponse._implicitHeader': 'https://github.com/nodejs/node/blob/HEAD/lib/_http_server.js#L406',
    'serverresponse.assignSocket': 'https://github.com/nodejs/node/blob/HEAD/lib/_http_server.js#L299',
    'serverresponse.detachSocket': 'https://github.com/nodejs/node/blob/HEAD/lib/_http_server.js#L310',
    'serverresponse.statusCode': 'https://github.com/nodejs/node/blob/HEAD/lib/_http_server.js#L272',

Performance estimate (single CI run)

  • Generation time: 24.8% faster (1.53 s → 1.15 s)
  • Peak memory: 1.9% lower (368.61 MB → 361.46 MB)

legacy-html Generator

Performance estimate (single CI run)

  • Generation time: 2.2% slower (19.00 s → 19.41 s)
  • Peak memory: 2.4% lower (2.38 GB → 2.32 GB)

legacy-json Generator

Performance estimate (single CI run)

  • Generation time: 3.6% faster (9.34 s → 9.00 s)
  • Peak memory: 4.2% higher (1.70 GB → 1.77 GB)

llms-txt Generator

Performance estimate (single CI run)

  • Generation time: 8.7% faster (8.78 s → 8.02 s)
  • Peak memory: 3.3% lower (1.79 GB → 1.74 GB)

orama-db Generator

Output size: 1 file changed · net +1.00 B

File size details
File Main PR Change
orama-db.json 8.89 MB 8.89 MB +1.00 B (+0.0%)

Performance estimate (single CI run)

  • Generation time: 4.7% faster (9.19 s → 8.76 s)
  • Peak memory: 1.7% higher (1.71 GB → 1.73 GB)

web Generator

Output size: 3 files changed · net +88.00 B

File size details
File Main PR Change
assets/all-FbBHLixS.js 22.13 MB -22.13 MB (-100.0%)
assets/all-m3n2S8pb.js 22.13 MB +22.13 MB
all.html 21.37 MB 21.37 MB +88.00 B (+0.0%)

Performance estimate (single CI run)

  • Generation time: 1.7% faster (75.90 s → 74.64 s)
  • Peak memory: 0.6% higher (5.20 GB → 5.23 GB)

Comment thread .changeset/specifier-generator-loading.md
@avivkeller
avivkeller requested a review from ovflowd July 30, 2026 23:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants