diff --git a/docs/ns-builtin-modules.md b/docs/ns-builtin-modules.md new file mode 100644 index 000000000..589b42acc --- /dev/null +++ b/docs/ns-builtin-modules.md @@ -0,0 +1,181 @@ +# `ns:` builtin modules + +Cross-runtime contract for exposing runtime-provided modules to application +code. This document is the specification both the iOS and Android runtimes +implement; a capability must behave identically on both platforms before it +ships in a stable release. + +## The scheme + +Builtin modules live under the URL-style `ns:` scheme, mirroring Node's +`node:` prefix: + +```js +// CommonJS +const util = require("ns:util"); + +// ES modules +import util, { inspect } from "ns:util"; +const util2 = await import("ns:util"); +``` + +Rules: + +- `ns:` specifiers are resolved by the runtime **before any filesystem or + npm resolution**. They can never be shadowed by a file, a path mapping, or + a package — and conversely, a file named `ns:util` is not reachable. +- Resolution of an unknown builtin fails synchronously with an `Error` whose + message is exactly `No such built-in module: ns:` (matching Node's + wording for familiarity). +- A builtin module is a **singleton per JS realm** (main context and each + worker get their own instance). `require("ns:util")` twice returns the same + object; the CJS exports object and the ESM namespace expose the same + underlying values (ESM additionally provides the exports object as + `default`). +- There is no bare-specifier fallback: `require("util")` is **not** an alias + for `require("ns:util")`. The unprefixed name continues to resolve to npm + packages as it always has. +- Builtin exports are frozen. Apps patch behavior by wrapping, not by + mutating the runtime's module. + +## Modules + +### `ns:util` (v1) + +| export | description | +|---|---| +| `inspect(value[, options])` | Formats any value for human consumption: depth-limited, output-capped, cycle-safe, never invokes getters (except a guarded `error.stack` read and custom `toString` overrides, which are honored). `options.depth` (number) overrides the default depth of 2. Other option keys are reserved. | +| `format(fmt, ...args)` | Node-style printf formatting: `%s`, `%d`, `%i`, `%f`, `%j`, `%o`, `%O`, `%%`. Extra arguments are appended space-separated, objects rendered via `inspect`. When `fmt` is not a string or contains no substitutions, all arguments are formatted and joined with spaces. `console.*` routes its arguments through this, so `console.log("%d apples", 3)` works. | + +**Stability caveat (verbatim from Node's contract):** the output of `inspect` +(and therefore `format`'s object rendering) may change between runtime +versions for readability; it is intended for humans and must not be parsed +programmatically. + +## `node:` compatibility shims + +The same registry serves the `node:` scheme with **compatibility shims** so +npm packages that require Node builtins by their prefixed names can run +unmodified where a shim exists: + +- A shim implements a documented **subset** of the corresponding Node module's + API, backed by `ns:` modules. Unimplemented members are simply absent + (so `typeof util.promisify === "function"` feature-checks behave + correctly); they are never present-but-throwing. +- **One source file per specifier.** A shim is its own module that consumes + the `ns:` module it adapts through the internal require, and it owns *all* + the adaptation — argument shapes, option names, aliases, anything that has + to track Node. A standard `ns:` module never contains compatibility code + and never knows a shim exists. +- Shims are **lazy**: a shim's source is only evaluated when its specifier is + first resolved, so an app that never touches the `node:` scheme never pays + for one. +- `node:` modules with no shim fail with the same error shape: + `No such built-in module: node:`. +- **Bare specifiers are untouched**: `require("util")` resolves through npm + as it always has (many apps bundle the `util` polyfill package). Only the + explicit `node:` prefix reaches the shim registry, so existing apps cannot + break. Bundler-level aliases (webpack/rollup) continue to work and take + precedence at build time. +- A shim is always a **distinct module object** from any `ns:` module, even + when every member is re-exported unchanged. `ns:` modules may grow runtime-specific + members freely; a `node:` shim only ever gains members that track Node's + actual API. This mirrors how Bun (`bun:*`), Deno (`Deno.*`/JSR) and + Cloudflare (`cloudflare:*`) all keep their own surface strictly apart from + their `node:` compat layer. +- Shims ship on both runtimes under the same parity rule as `ns:` modules. + +### v1 shims + +| module | exports | notes | +|---|---|---| +| `node:util` | `inspect`, `format` | Re-exports `ns:util`'s members unchanged (`nodeUtil.inspect === nsUtil.inspect`) from a **distinct, separately frozen module object**. Documented as partial. | + +Candidates for future shims, in rough order of ecosystem demand: +`node:events` (EventEmitter), `node:path` (pure JS), `node:buffer`, +`node:process` (subset). Each requires a spec update here first. + +## The internal require + +Builtin modules reach each other — and only each other — through an internal +`require` the runtime provides to every builtin source: + +- It resolves **builtin specifiers only**. A path, a package name or any other + specifier is not reachable from a builtin; an unregistered builtin name + throws the same `No such built-in module: ` an app sees. +- It materializes the target module on first use and returns the realm's + singleton afterwards, which is what makes shims lazy. +- Requiring a module that is still being built throws rather than recursing, + so a dependency cycle between builtins is a loud error and not a hang. + +This is the mechanism shims are built on, so it is normative: both runtimes +provide it. + +## Adding a builtin module + +- The name is a short, lowercase identifier (`ns:util`, `ns:timers`, ...). +- One module, one source file, one registry entry — including shims. +- New modules and new exports require this document to be updated first and + an implementation on both runtimes before a stable release; a module may + ship on one platform behind a documented "experimental, iOS-only" (or + Android-only) note in between. +- Internal runtime machinery must never be reachable through the scheme: + the registry distinguishes public modules from internal builtins, and only + public ones resolve (Node's `canBeRequiredByUsers` split). + +## Source-text modules: deliberately not supported + +Builtins are classic function bodies, not ES modules, on both runtimes. If +cross-builtin code sharing is ever needed, the first answer is bundling at +generation time (author as ESM, emit function bodies); runtime source-text +builtin modules (Node's `kSourceTextModule`) are justified only by a concrete +need for live module semantics (TLA, live bindings, cyclic imports), which no +current or planned builtin has. Revisit here before building either. + +## iOS implementation notes (non-normative) + +Builtin modules are function-body builtins (`NativeScript/runtime/js/`, +see the README there) compiled via the RuntimeBuiltins table. The `ns:` +resolver intercepts specifiers in the CommonJS require path and in the ES +module resolve/dynamic-import callbacks; ESM consumption is served by a +synthetic module whose exports are populated from the same per-realm exports +object. The internal require is a fixed parameter of the builtin function +wrapper (`exports`, `require`, `module`, `binding`, `primordials`). + +iOS also keeps a pre-registry `node:url` polyfill (`fileURLToPath`, +`pathToFileURL`) that predates this document. It is compiled from module +source inside the ES module resolver and is therefore reachable through +`import` only, not through `require()`. + +## Android implementation notes (non-normative) + +Builtin modules are function-body builtins +(`test-app/runtime/src/main/cpp/js/`, see the README there) compiled via the +RuntimeBuiltins table. The registry lives in +`test-app/runtime/src/main/cpp/NsBuiltinModules.{h,cpp}` and intercepts +specifiers in the CommonJS require path (`ModuleInternal::RequireCallbackImpl`) +and in the ES module resolve and dynamic-import callbacks +(`ModuleInternalCallbacks.cpp`); ESM consumption is served by a synthetic +module whose exports are populated from the same per-realm exports object. The +internal require is a fixed parameter of the builtin function wrapper +(`exports`, `require`, `module`, `binding`, `primordials`). + +Android has no `Caches` class, so every per-realm cache — exports objects, +synthetic modules, the in-progress set, the cached `format` and the builtin +`require` — lives in an isolate-keyed map released from `disposeIsolate`. +The ES module registry app modules land in (`g_moduleRegistry`) is +process-global and shared by every isolate; the builtin caches deliberately do +not use it, so workers get their own instances as the spec requires. + +Android also keeps three pre-registry `node:` polyfills that predate this +document: `node:url` (`fileURLToPath`, `pathToFileURL`), `node:module` +(`createRequire`) and `node:path` (`sep`, `delimiter`, `basename`, `dirname`, +`extname`, `join`, `resolve`, `isAbsolute`). They are compiled from module +source inside the ES module resolver and are therefore reachable through +`import` only; `require("node:path")` reaches the registry and fails with the +not-found message. + +There is deliberately no `node:fs` polyfill and no catch-all for unshimmed +`node:` names: a stub whose members throw on use violates the +absent-not-present-but-throwing rule above, and an empty-default fallback lets +an import that cannot work succeed. diff --git a/eslint.config.mjs b/eslint.config.mjs index 9c042cef0..82c40c2f7 100644 --- a/eslint.config.mjs +++ b/eslint.config.mjs @@ -1,8 +1,8 @@ // Lint setup for the runtime's builtin JavaScript // (test-app/runtime/src/main/cpp/js). Each file is compiled by BuiltinLoader -// as a FUNCTION BODY with the fixed parameters `exports`, `module`, `binding` -// and `primordials` (see that directory's README.md), which are declared as -// globals here. no-undef is the typo net for binding-bag destructures and +// as a FUNCTION BODY with the fixed parameters `exports`, `require`, `module`, +// `binding` and `primordials` (see that directory's README.md), which are +// declared as globals here. no-undef is the typo net for binding-bag destructures and // native-global usage alike; no-restricted-properties keeps the captured // intrinsics from being read off the live globals again. import globals from 'globals'; @@ -15,8 +15,11 @@ const capturedStatics = [ ['Array', 'isArray', 'ArrayIsArray'], ['ArrayBuffer', 'isView', 'ArrayBufferIsView'], ['JSON', 'stringify', 'JSONStringify'], + ['Number', 'parseFloat', 'NumberParseFloat'], + ['Number', 'parseInt', 'NumberParseInt'], ['Object', 'create', 'ObjectCreate'], ['Object', 'defineProperty', 'ObjectDefineProperty'], + ['Object', 'freeze', 'ObjectFreeze'], ['Object', 'getOwnPropertyDescriptor', 'ObjectGetOwnPropertyDescriptor'], ['Object', 'getOwnPropertySymbols', 'ObjectGetOwnPropertySymbols'], ['Object', 'getPrototypeOf', 'ObjectGetPrototypeOf'], @@ -26,7 +29,7 @@ const capturedStatics = [ // Captured constructors. A destructure from `primordials` shadows the global, // so these only fire on the unguarded reference. -const restrictedGlobals = ['Date', 'Map', 'Proxy', 'Set', 'String', 'TypeError'].map((name) => ({ +const restrictedGlobals = ['Date', 'Map', 'Number', 'Proxy', 'Set', 'String', 'TypeError'].map((name) => ({ name, message: `Destructure ${name} from primordials — builtins must not read intrinsics off globals user code can replace.`, })); @@ -46,6 +49,7 @@ export default [ globals: { ...globals.es2021, exports: 'readonly', + require: 'readonly', module: 'readonly', binding: 'readonly', primordials: 'readonly', diff --git a/test-app/app/src/main/assets/app/mainpage.js b/test-app/app/src/main/assets/app/mainpage.js index 7e37e0869..ca13a904d 100644 --- a/test-app/app/src/main/assets/app/mainpage.js +++ b/test-app/app/src/main/assets/app/mainpage.js @@ -81,6 +81,8 @@ require('./tests/testUncaughtErrorPolicy'); // Runtime builtins keep working when app code replaces the intrinsics they use require('./tests/testPrimordials'); require('./tests/testInspect'); +// The ns:/node: builtin modules +require('./tests/testNsUtil'); require("./tests/testConcurrentAccess"); require("./tests/testESModules.mjs"); diff --git a/test-app/app/src/main/assets/app/testNsUtilImport.mjs b/test-app/app/src/main/assets/app/testNsUtilImport.mjs new file mode 100644 index 000000000..00cb7b01f --- /dev/null +++ b/test-app/app/src/main/assets/app/testNsUtilImport.mjs @@ -0,0 +1,6 @@ +// Static `import` of the builtins, so the ES module resolve callback is +// exercised too (dynamic import() has its own fast path). +import nsDefault, { inspect, format } from "ns:util"; +import nodeDefault, { inspect as nodeInspect } from "node:util"; + +export { nsDefault, inspect, format, nodeDefault, nodeInspect }; diff --git a/test-app/app/src/main/assets/app/tests/nsUtilWorker.js b/test-app/app/src/main/assets/app/tests/nsUtilWorker.js new file mode 100644 index 000000000..afba289fb --- /dev/null +++ b/test-app/app/src/main/assets/app/tests/nsUtilWorker.js @@ -0,0 +1,10 @@ +// A builtin module is a singleton per realm, so the worker builds its own +// ns:util rather than reaching the main context's instance. +onmessage = function () { + var util = require("ns:util"); + postMessage({ + formatted: util.format("%d apples", 3), + singleton: require("ns:util") === util, + frozen: Object.isFrozen(util), + }); +}; diff --git a/test-app/app/src/main/assets/app/tests/testNsUtil.js b/test-app/app/src/main/assets/app/tests/testNsUtil.js new file mode 100644 index 000000000..acf05c0ff --- /dev/null +++ b/test-app/app/src/main/assets/app/tests/testNsUtil.js @@ -0,0 +1,204 @@ +describe("ns:util", function () { + var util = require("ns:util"); + + it("exports a frozen inspect/format pair", function () { + expect(Object.isFrozen(util)).toBe(true); + expect(typeof util.inspect).toBe("function"); + expect(typeof util.format).toBe("function"); + expect(util.inspect({ a: 1 })).toBe("{ a: 1 }"); + }); + + it("is a singleton per realm", function () { + expect(require("ns:util")).toBe(util); + }); + + it("throws for an unknown builtin", function () { + expect(function () { + require("ns:definitely-not-a-module"); + }).toThrow(); + + var error; + try { + require("ns:definitely-not-a-module"); + } catch (e) { + error = e; + } + expect(error instanceof Error).toBe(true); + expect(error.message).toBe("No such built-in module: ns:definitely-not-a-module"); + }); + + it("leaves bare specifiers to npm resolution", function () { + var resolved; + try { + resolved = require("util"); + } catch (e) { + resolved = undefined; + } + expect(resolved).not.toBe(util); + expect(resolved && resolved.format).not.toBe(util.format); + }); + + describe("format", function () { + var format = util.format; + + it("substitutes %s, %d, %i and %f", function () { + expect(format("%s", "a")).toBe("a"); + expect(format("%s", 3)).toBe("3"); + expect(format("%s", { a: 1 })).toBe("{ a: 1 }"); + expect(format("%d apples", 3)).toBe("3 apples"); + expect(format("%d", "4.7")).toBe("4.7"); + expect(format("%i", "4.7")).toBe("4"); + expect(format("%f", "4.7")).toBe("4.7"); + }); + + it("substitutes %j, %o and %O", function () { + expect(format("%j", { a: 1 })).toBe('{"a":1}'); + expect(format("%o", { a: 1 })).toBe("{ a: 1 }"); + expect(format("%O", { a: 1 })).toBe("{ a: 1 }"); + }); + + it("renders a circular value for %j as [Circular]", function () { + var circular = {}; + circular.self = circular; + expect(format("%j", circular)).toBe("[Circular]"); + }); + + it("collapses %% and leaves unknown or dangling % verbatim", function () { + expect(format("100%")).toBe("100%"); + expect(format("100%", "x")).toBe("100% x"); + expect(format("%% done", 1)).toBe("% done 1"); + expect(format("a %x b", 1)).toBe("a %x b 1"); + expect(format("trailing %", 1)).toBe("trailing % 1"); + expect(format("%s %s", "a")).toBe("a %s"); + }); + + it("appends extra arguments space-separated", function () { + expect(format("%s", "a", "b", { c: 1 })).toBe("a b { c: 1 }"); + }); + + it("joins every argument when the first is not a format string", function () { + expect(format({ a: 1 }, "x")).toBe("{ a: 1 } x"); + expect(format(1, 2)).toBe("1 2"); + expect(format()).toBe(""); + }); + }); + + it("backs console.* formatting", function () { + expect(function () { + console.log("%d apples", 3); + console.log("100%"); + }).not.toThrow(); + }); + + it("resolves through dynamic import with the same members", function (done) { + import("ns:util").then(function (ns) { + expect(ns.default).toBe(util); + expect(ns.inspect).toBe(util.inspect); + expect(ns.format).toBe(util.format); + // The synthetic module is cached per realm. + return import("ns:util").then(function (again) { + expect(again).toBe(ns); + done(); + }); + }, function (error) { + fail("import('ns:util') should resolve: " + error); + done(); + }); + }); + + it("rejects a dynamic import of an unknown builtin", function (done) { + import("ns:definitely-not-a-module").then(function () { + fail("import of an unknown builtin should reject"); + done(); + }, function (error) { + expect(error instanceof Error).toBe(true); + expect(error.message).toBe("No such built-in module: ns:definitely-not-a-module"); + done(); + }); + }); + + it("resolves through a static import", function (done) { + import("~/testNsUtilImport.mjs").then(function (module) { + expect(module.nsDefault).toBe(util); + expect(module.inspect).toBe(util.inspect); + expect(module.format).toBe(util.format); + expect(module.nodeDefault).toBe(require("node:util")); + expect(module.nodeInspect).toBe(util.inspect); + done(); + }, function (error) { + fail("static import of ns:util should resolve: " + error); + done(); + }); + }); + + it("gives a worker its own instance", function (done) { + var worker = new Worker("./nsUtilWorker.js"); + worker.onmessage = function (msg) { + expect(msg.data.formatted).toBe("3 apples"); + expect(msg.data.singleton).toBe(true); + expect(msg.data.frozen).toBe(true); + worker.terminate(); + done(); + }; + worker.onerror = function (error) { + fail("worker failed: " + error.message); + worker.terminate(); + done(); + }; + worker.postMessage("go"); + }); +}); + +describe("node:util", function () { + var util = require("ns:util"); + var nodeUtil = require("node:util"); + + it("is a distinct module object sharing ns:util's members", function () { + expect(nodeUtil).not.toBe(util); + expect(Object.isFrozen(nodeUtil)).toBe(true); + expect(nodeUtil.inspect).toBe(util.inspect); + expect(nodeUtil.format).toBe(util.format); + }); + + it("is a singleton per realm", function () { + expect(require("node:util")).toBe(nodeUtil); + }); + + it("only exposes what the shim implements", function () { + expect(typeof nodeUtil.promisify).toBe("undefined"); + }); + + it("throws for a node builtin with no shim", function () { + var error; + try { + require("node:fs"); + } catch (e) { + error = e; + } + expect(error instanceof Error).toBe(true); + expect(error.message).toBe("No such built-in module: node:fs"); + }); + + it("rejects a dynamic import of a node builtin with no shim", function (done) { + import("node:fs").then(function () { + fail("import of an unshimmed node builtin should reject"); + done(); + }, function (error) { + expect(error instanceof Error).toBe(true); + expect(error.message).toBe("No such built-in module: node:fs"); + done(); + }); + }); + + it("resolves through dynamic import with the same members", function (done) { + import("node:util").then(function (ns) { + expect(ns.default).toBe(nodeUtil); + expect(ns.inspect).toBe(util.inspect); + expect(ns.format).toBe(util.format); + done(); + }, function (error) { + fail("import('node:util') should resolve: " + error); + done(); + }); + }); +}); diff --git a/test-app/runtime/CMakeLists.txt b/test-app/runtime/CMakeLists.txt index e45a338e8..edf16f966 100644 --- a/test-app/runtime/CMakeLists.txt +++ b/test-app/runtime/CMakeLists.txt @@ -70,6 +70,8 @@ set(RUNTIME_BUILTIN_JS ${RUNTIME_BUILTIN_JS_DIR}/inspect.js ${RUNTIME_BUILTIN_JS_DIR}/json-helper.js ${RUNTIME_BUILTIN_JS_DIR}/message-loop-timer.js + ${RUNTIME_BUILTIN_JS_DIR}/node-util.js + ${RUNTIME_BUILTIN_JS_DIR}/ns-util.js ${RUNTIME_BUILTIN_JS_DIR}/primordials.js ${RUNTIME_BUILTIN_JS_DIR}/require-factory.js ${RUNTIME_BUILTIN_JS_DIR}/weak-ref.js @@ -167,6 +169,7 @@ add_library( src/main/cpp/ModuleInternal.cpp src/main/cpp/ModuleInternalCallbacks.cpp src/main/cpp/NativeScriptException.cpp + src/main/cpp/NsBuiltinModules.cpp src/main/cpp/NumericCasts.cpp src/main/cpp/ObjectManager.cpp src/main/cpp/Profiler.cpp diff --git a/test-app/runtime/src/main/cpp/BuiltinLoader.cpp b/test-app/runtime/src/main/cpp/BuiltinLoader.cpp index c276303bf..b0ad55ced 100644 --- a/test-app/runtime/src/main/cpp/BuiltinLoader.cpp +++ b/test-app/runtime/src/main/cpp/BuiltinLoader.cpp @@ -4,6 +4,7 @@ #include #include "ArgConverter.h" +#include "NsBuiltinModules.h" #include "robin_hood.h" using namespace v8; @@ -22,22 +23,74 @@ std::vector builtinCache[static_cast(BuiltinId::kCount)]; /* * Every builtin is compiled as a function body receiving these fixed * parameters, mirroring Node's module wrapper: a file exports through - * `module.exports`/`exports`, natives arrive as properties of the `binding` - * bag (Node's internalBinding idiom) and intrinsics as properties of - * `primordials`; each file destructures what it needs. + * `module.exports`/`exports`, reaches sibling builtin modules through + * `require`, natives arrive as properties of the `binding` bag (Node's + * internalBinding idiom) and intrinsics as properties of `primordials`; each + * file destructures what it needs. */ constexpr const char* kExportsParamName = "exports"; +constexpr const char* kRequireParamName = "require"; constexpr const char* kModuleParamName = "module"; constexpr const char* kBindingParamName = "binding"; constexpr const char* kPrimordialsParamName = "primordials"; -constexpr size_t kParamCount = 4; +constexpr size_t kParamCount = 5; /* - * Per-isolate intrinsics snapshot. Worker runtimes initialize on their own - * threads, so every access is under the mutex. + * Per-isolate intrinsics snapshot and builtin require. Worker runtimes + * initialize on their own threads, so every access is under the mutex. */ std::mutex primordialsMutex; robin_hood::unordered_map*> isolateToPrimordials; +std::mutex builtinRequireMutex; +robin_hood::unordered_map*> isolateToBuiltinRequire; + +/* + * The `require` every builtin receives: builtin specifiers only, so a builtin + * can never reach application code or the filesystem. + */ +void BuiltinRequireCallback(const FunctionCallbackInfo& info) { + Isolate* isolate = info.GetIsolate(); + if (info.Length() < 1 || !info[0]->IsString()) { + isolate->ThrowException(Exception::TypeError(ArgConverter::ConvertToV8String( + isolate, "require() expects a specifier string"))); + return; + } + + Local context = isolate->GetCurrentContext(); + std::string specifier = ArgConverter::ConvertToString(info[0].As()); + Local exports; + if (NsBuiltinModules::GetExports(context, specifier).ToLocal(&exports)) { + info.GetReturnValue().Set(exports); + } else if (!NsBuiltinModules::IsRegistered(specifier)) { + isolate->ThrowException(Exception::Error(ArgConverter::ConvertToV8String( + isolate, NsBuiltinModules::NotFoundMessage(specifier)))); + } +} + +MaybeLocal GetBuiltinRequire(Local context) { + Isolate* isolate = v8::Isolate::GetCurrent(); + + { + std::lock_guard lock(builtinRequireMutex); + auto it = isolateToBuiltinRequire.find(isolate); + if (it != isolateToBuiltinRequire.end()) { + return it->second->Get(isolate); + } + } + + Local require; + if (!v8::Function::New(context, BuiltinRequireCallback, Local(), 1, + ConstructorBehavior::kThrow) + .ToLocal(&require)) { + return MaybeLocal(); + } + + { + std::lock_guard lock(builtinRequireMutex); + isolateToBuiltinRequire.emplace(isolate, new Persistent(isolate, require)); + } + return require; +} MaybeLocal CompileBuiltin(Local context, BuiltinId id) { Isolate* isolate = v8::Isolate::GetCurrent(); @@ -57,6 +110,7 @@ MaybeLocal CompileBuiltin(Local context, BuiltinId id) { isolate, builtin.source, static_cast(builtin.length)); Local params[] = { ArgConverter::ConvertToV8String(isolate, kExportsParamName), + ArgConverter::ConvertToV8String(isolate, kRequireParamName), ArgConverter::ConvertToV8String(isolate, kModuleParamName), ArgConverter::ConvertToV8String(isolate, kBindingParamName), ArgConverter::ConvertToV8String(isolate, kPrimordialsParamName)}; @@ -106,6 +160,11 @@ MaybeLocal CallBuiltin(Local context, BuiltinId id, Local return MaybeLocal(); } + Local require; + if (!GetBuiltinRequire(context).ToLocal(&require)) { + return MaybeLocal(); + } + Local exportsObj = Object::New(isolate); Local moduleObj = Object::New(isolate); Local exportsKey = ArgConverter::ConvertToV8String(isolate, kExportsParamName); @@ -113,7 +172,7 @@ MaybeLocal CallBuiltin(Local context, BuiltinId id, Local return MaybeLocal(); } - Local args[] = {exportsObj, moduleObj, + Local args[] = {exportsObj, require, moduleObj, binding.IsEmpty() ? Undefined(isolate).As() : binding, primordials}; if (fn->Call(context, Undefined(isolate), static_cast(kParamCount), args).IsEmpty()) { @@ -168,11 +227,20 @@ MaybeLocal BuiltinLoader::RunBuiltin(Local context, BuiltinId id } void BuiltinLoader::onDisposeIsolate(Isolate* isolate) { - std::lock_guard lock(primordialsMutex); - auto it = isolateToPrimordials.find(isolate); - if (it != isolateToPrimordials.end()) { + { + std::lock_guard lock(primordialsMutex); + auto it = isolateToPrimordials.find(isolate); + if (it != isolateToPrimordials.end()) { + delete it->second; + isolateToPrimordials.erase(it); + } + } + + std::lock_guard lock(builtinRequireMutex); + auto it = isolateToBuiltinRequire.find(isolate); + if (it != isolateToBuiltinRequire.end()) { delete it->second; - isolateToPrimordials.erase(it); + isolateToBuiltinRequire.erase(it); } } diff --git a/test-app/runtime/src/main/cpp/BuiltinLoader.h b/test-app/runtime/src/main/cpp/BuiltinLoader.h index 89c242a5c..4a202c8f9 100644 --- a/test-app/runtime/src/main/cpp/BuiltinLoader.h +++ b/test-app/runtime/src/main/cpp/BuiltinLoader.h @@ -10,12 +10,14 @@ class BuiltinLoader { public: /* * Compiles the builtin identified by id as a function body with the fixed - * parameters `exports`, `module`, `binding` (Node's module wrapper plus - * its internalBinding idiom) and `primordials`, calls it with the given - * bag of natives (or undefined when omitted) plus this isolate's frozen - * intrinsics snapshot, and returns the resulting `module.exports`. The - * snapshot is produced by the kPrimordials builtin on first use and cached - * per isolate, so it is taken before any user code can replace a global. + * parameters `exports`, `require`, `module`, `binding` (Node's module + * wrapper plus its internalBinding idiom) and `primordials`, calls it with + * the given bag of natives (or undefined when omitted) plus this isolate's + * frozen intrinsics snapshot, and returns the resulting `module.exports`. + * `require` reaches the builtin modules (NsBuiltinModules) and nothing + * else. The snapshot is produced by the kPrimordials builtin on first use + * and cached per isolate, so it is taken before any user code can replace + * a global. * Scripts carry an "internal/.js" origin so runtime frames are * identifiable in stack traces. Compilation goes through a process-wide * bytecode cache: the first run in the process compiles eagerly and diff --git a/test-app/runtime/src/main/cpp/IsolateDisposer.cpp b/test-app/runtime/src/main/cpp/IsolateDisposer.cpp index 8a2aa776b..fede75cad 100644 --- a/test-app/runtime/src/main/cpp/IsolateDisposer.cpp +++ b/test-app/runtime/src/main/cpp/IsolateDisposer.cpp @@ -7,6 +7,7 @@ #include "BuiltinLoader.h" #include "JSONObjectHelper.h" #include "MetadataNode.h" +#include "NsBuiltinModules.h" #include "V8GlobalHelpers.h" #include @@ -18,6 +19,7 @@ namespace tns { tns::MetadataNode::onDisposeIsolate(isolate); tns::Console::onDisposeIsolate(isolate); tns::JSONObjectHelper::onDisposeIsolate(isolate); + tns::NsBuiltinModules::onDisposeIsolate(isolate); tns::BuiltinLoader::onDisposeIsolate(isolate); // clear all isolate bound objects std::lock_guard lock(isolateBoundObjectsMutex_); diff --git a/test-app/runtime/src/main/cpp/ModuleInternal.cpp b/test-app/runtime/src/main/cpp/ModuleInternal.cpp index dec5a0465..b4c221f6b 100644 --- a/test-app/runtime/src/main/cpp/ModuleInternal.cpp +++ b/test-app/runtime/src/main/cpp/ModuleInternal.cpp @@ -14,6 +14,7 @@ #include "NativeScriptAssert.h" #include "Constants.h" #include "NativeScriptException.h" +#include "NsBuiltinModules.h" #include "Util.h" #include "SimpleProfiler.h" #include "include/v8.h" @@ -174,6 +175,23 @@ void ModuleInternal::RequireCallbackImpl(const v8::FunctionCallbackInfo()); + + // Builtin modules resolve before any path handling, so they can never be + // shadowed by a file or a package, and an unknown one fails as a missing + // builtin rather than as a missing file. Only prefixed specifiers get + // here: a bare `util` still resolves through npm. + if (NsBuiltinModules::IsBuiltinScheme(moduleName)) { + auto context = isolate->GetCurrentContext(); + Local exports; + if (NsBuiltinModules::GetExports(context, moduleName).ToLocal(&exports)) { + args.GetReturnValue().Set(exports); + } else if (!NsBuiltinModules::IsRegistered(moduleName)) { + isolate->ThrowException(Exception::Error(ArgConverter::ConvertToV8String( + isolate, NsBuiltinModules::NotFoundMessage(moduleName)))); + } + return; + } + tns::instrumentation::Frame frame("RequireCallback " + moduleName); string callingModuleDirName = ArgConverter::ConvertToString(args[1].As()); auto isData = false; diff --git a/test-app/runtime/src/main/cpp/ModuleInternalCallbacks.cpp b/test-app/runtime/src/main/cpp/ModuleInternalCallbacks.cpp index 2ff32b57e..7c2f2770d 100644 --- a/test-app/runtime/src/main/cpp/ModuleInternalCallbacks.cpp +++ b/test-app/runtime/src/main/cpp/ModuleInternalCallbacks.cpp @@ -2,6 +2,7 @@ #include "ArgConverter.h" #include "NativeScriptException.h" #include "NativeScriptAssert.h" +#include "NsBuiltinModules.h" #include "Runtime.h" #include "Util.h" #include @@ -344,6 +345,20 @@ v8::MaybeLocal ResolveModuleCallback(v8::Local context, DEBUG_WRITE("ResolveModuleCallback: Resolving '%s'", spec.c_str()); } + // Builtin modules resolve before any path handling. Unshimmed "node:" + // names fall through to the legacy polyfills below. + if (NsBuiltinModules::IsRegistered(spec) || NsBuiltinModules::IsNsScheme(spec)) { + v8::Local builtin; + if (NsBuiltinModules::GetModule(context, spec).ToLocal(&builtin)) { + return v8::MaybeLocal(builtin); + } + if (!NsBuiltinModules::IsRegistered(spec)) { + isolate->ThrowException(v8::Exception::Error( + ArgConverter::ConvertToV8String(isolate, NsBuiltinModules::NotFoundMessage(spec)))); + } + return v8::MaybeLocal(); + } + // Normalize malformed http:/ and https:/ prefixes if (spec.rfind("http:/", 0) == 0 && spec.rfind("http://", 0) != 0) { spec.insert(5, "/"); @@ -691,29 +706,10 @@ v8::MaybeLocal ResolveModuleCallback(v8::Local context, "}\n" "\n" "export default { basename, dirname, extname, join, resolve, isAbsolute, sep, delimiter };\n"; - } else if (builtinName == "fs") { - // Create a basic polyfill for node:fs - polyfillContent = "// Polyfill for node:fs\n" - "console.warn('Node.js fs module is not supported in NativeScript. Use @nativescript/core File APIs instead.');\n" - "\n" - "export function readFileSync() {\n" - " throw new Error('fs.readFileSync is not supported in NativeScript. Use @nativescript/core File APIs.');\n" - "}\n" - "\n" - "export function writeFileSync() {\n" - " throw new Error('fs.writeFileSync is not supported in NativeScript. Use @nativescript/core File APIs.');\n" - "}\n" - "\n" - "export function existsSync() {\n" - " throw new Error('fs.existsSync is not supported in NativeScript. Use @nativescript/core File APIs.');\n" - "}\n" - "\n" - "export default { readFileSync, writeFileSync, existsSync };\n"; } else { - // Generic polyfill for other Node.js built-in modules - polyfillContent = "// Polyfill for node:" + builtinName + "\n" - "console.warn('Node.js built-in module \\'node:" + builtinName + "\\' is not fully supported in NativeScript');\n" - "export default {};\n"; + isolate->ThrowException(v8::Exception::Error( + ArgConverter::ConvertToV8String(isolate, NsBuiltinModules::NotFoundMessage(spec)))); + return v8::MaybeLocal(); } // Create module source and compile it in-memory @@ -880,6 +876,23 @@ v8::MaybeLocal ImportModuleDynamicallyCallback( return v8::MaybeLocal(); } + // Builtin modules never reach the loader below; the namespace comes + // straight from the realm's synthetic module. + if (NsBuiltinModules::IsRegistered(spec) || NsBuiltinModules::IsNsScheme(spec)) { + v8::TryCatch tc(isolate); + v8::Local builtin; + if (NsBuiltinModules::GetModule(context, spec).ToLocal(&builtin)) { + resolver->Resolve(context, builtin->GetModuleNamespace()).FromMaybe(false); + } else { + v8::Local error = + tc.HasCaught() ? tc.Exception() + : v8::Exception::Error(ArgConverter::ConvertToV8String( + isolate, NsBuiltinModules::NotFoundMessage(spec))); + resolver->Reject(context, error).FromMaybe(false); + } + return scope.Escape(resolver->GetPromise()); + } + // Resolve relative or root-absolute dynamic imports against the referrer's URL when provided auto isHttpLike = [](const std::string& s) -> bool { return s.rfind("http://", 0) == 0 || s.rfind("https://", 0) == 0; @@ -984,19 +997,29 @@ v8::MaybeLocal ImportModuleDynamicallyCallback( // ScriptOrModule, and the specifier above is already absolute when needed. v8::Local refMod; - v8::MaybeLocal maybeModule = - ResolveModuleCallback(context, resolvedSpecifier, import_assertions, refMod); - v8::Local module; - if (!maybeModule.ToLocal(&module)) { - // Resolution failed; reject to avoid leaving a pending Promise (white screen) - if (IsScriptLoadingLogEnabled()) { - DEBUG_WRITE("ImportModuleDynamicallyCallback: Resolution failed for '%s'", spec.c_str()); + { + v8::TryCatch resolveTc(isolate); + v8::MaybeLocal maybeModule = + ResolveModuleCallback(context, resolvedSpecifier, import_assertions, refMod); + + if (!maybeModule.ToLocal(&module)) { + // Resolution failed; reject to avoid leaving a pending Promise (white screen) + if (IsScriptLoadingLogEnabled()) { + DEBUG_WRITE("ImportModuleDynamicallyCallback: Resolution failed for '%s'", spec.c_str()); + } + // The resolver's own error carries the reason (a missing + // builtin names the exact contract message); only invent one + // when resolution failed without throwing. + v8::Local ex = + resolveTc.HasCaught() + ? resolveTc.Exception() + : v8::Exception::Error(ArgConverter::ConvertToV8String( + isolate, std::string("Failed to resolve module: ") + spec)); + resolveTc.Reset(); + resolver->Reject(context, ex).Check(); + return scope.Escape(resolver->GetPromise()); } - v8::Local ex = v8::Exception::Error( - ArgConverter::ConvertToV8String(isolate, std::string("Failed to resolve module: ") + spec)); - resolver->Reject(context, ex).Check(); - return scope.Escape(resolver->GetPromise()); } // If not yet instantiated/evaluated, do it now diff --git a/test-app/runtime/src/main/cpp/NsBuiltinModules.cpp b/test-app/runtime/src/main/cpp/NsBuiltinModules.cpp new file mode 100644 index 000000000..e1b43ddce --- /dev/null +++ b/test-app/runtime/src/main/cpp/NsBuiltinModules.cpp @@ -0,0 +1,350 @@ +#include "NsBuiltinModules.h" + +#include + +#include +#include + +#include "ArgConverter.h" +#include "BuiltinLoader.h" +#include "console/Console.h" +#include "robin_hood.h" + +using namespace v8; + +namespace tns { + +namespace { + +constexpr const char* kNsPrefix = "ns:"; +constexpr const char* kNodePrefix = "node:"; + +struct Registration { + const char* specifier; + BuiltinId builtin; +}; + +/* + * The v1 registry (docs/ns-builtin-modules.md). One specifier, one source + * file: a `node:` shim is its own builtin that requires the `ns:` module it + * adapts, so the two module objects stay distinct and the standard module + * never carries compatibility code. + */ +constexpr Registration kRegistry[] = { + {"ns:util", BuiltinId::kNsUtil}, + {"node:util", BuiltinId::kNodeUtil}, +}; + +const Registration* Find(const std::string& specifier) { + for (const Registration& registration : kRegistry) { + if (specifier == registration.specifier) { + return ®istration; + } + } + return nullptr; +} + +bool HasPrefix(const std::string& specifier, const char* prefix) { + return specifier.rfind(prefix, 0) == 0; +} + +/* + * A builtin module is a singleton per realm, so every cache here is per + * isolate: workers get their own exports objects and their own synthetic + * modules. The process-global g_moduleRegistry deliberately holds none of + * this. Worker runtimes initialize on their own threads, so the map itself is + * under the mutex; the state it holds is only ever touched from its isolate's + * thread. + */ +struct RealmState { + robin_hood::unordered_map*> exports; + robin_hood::unordered_map*> modules; + // Specifiers currently being built, so a shim requiring back into the + // module that is loading it fails instead of recursing. + robin_hood::unordered_set inProgress; + Persistent* format = nullptr; + bool formatUnavailable = false; + + ~RealmState() { + for (auto& entry : exports) { + delete entry.second; + } + for (auto& entry : modules) { + delete entry.second; + } + delete format; + } +}; + +std::mutex realmsMutex; +robin_hood::unordered_map isolateToRealm; + +RealmState& GetRealm(Isolate* isolate) { + std::lock_guard lock(realmsMutex); + auto it = isolateToRealm.find(isolate); + if (it == isolateToRealm.end()) { + it = isolateToRealm.emplace(isolate, new RealmState()).first; + } + return *it->second; +} + +MaybeLocal BuildBinding(Local context, BuiltinId builtin) { + Isolate* isolate = v8::Isolate::GetCurrent(); + Local binding = Object::New(isolate); + + switch (builtin) { + case BuiltinId::kNsUtil: { + // The console formatter is built once per realm; ns:util + // re-exports that instance instead of creating a second one. + Local inspect = Console::getInspect(context); + if (inspect.IsEmpty()) { + return MaybeLocal(); + } + if (!binding->Set(context, ArgConverter::ConvertToV8String(isolate, "inspect"), inspect) + .FromMaybe(false)) { + return MaybeLocal(); + } + break; + } + default: + break; + } + + return binding; +} + +/* + * Runs a module's builtin and caches its exports. Always leaves an exception + * pending when it returns false. + */ +bool Instantiate(Local context, const Registration& requested) { + Isolate* isolate = v8::Isolate::GetCurrent(); + RealmState& realm = GetRealm(isolate); + + /* + * A shim reaches its ns: module through the builtin require, so the graph + * is walked while a module is still being built; a cycle would otherwise + * recurse until the stack runs out. + */ + if (realm.inProgress.count(requested.specifier) > 0) { + isolate->ThrowException(Exception::Error(ArgConverter::ConvertToV8String( + isolate, "Circular require of built-in module: " + + std::string(requested.specifier)))); + return false; + } + realm.inProgress.emplace(requested.specifier); + + TryCatch tc(isolate); + Local binding; + Local result; + bool built = BuildBinding(context, requested.builtin).ToLocal(&binding) && + BuiltinLoader::RunBuiltin(context, requested.builtin, binding).ToLocal(&result) && + result->IsObject(); + realm.inProgress.erase(requested.specifier); + + if (built) { + realm.exports[requested.specifier] = new Persistent(isolate, result.As()); + return true; + } + + if (tc.HasCaught()) { + tc.ReThrow(); + return false; + } + isolate->ThrowException(Exception::Error(ArgConverter::ConvertToV8String( + isolate, "Failed to initialize built-in module '" + std::string(requested.specifier) + + "'"))); + return false; +} + +/* + * The exports a synthetic module re-exports by name, in the order used both + * when declaring the export names and when populating them. + */ +MaybeLocal ExportKeys(Local context, Local exports) { + return exports->GetOwnPropertyNames(context, PropertyFilter::ONLY_ENUMERABLE, + KeyConversionMode::kConvertToString); +} + +MaybeLocal NoDependencies(Local context, Local specifier, + Local import_attributes, Local referrer) { + // Synthetic modules never request anything. + return MaybeLocal(); +} + +MaybeLocal PopulateSyntheticModule(Local context, Local module) { + Isolate* isolate = v8::Isolate::GetCurrent(); + // The specifier was handed to CreateSyntheticModule as the module name, + // which is how these steps find the exports object again. + std::string specifier = ArgConverter::ConvertToString(module->GetResourceName().As()); + + Local exports; + if (!NsBuiltinModules::GetExports(context, specifier).ToLocal(&exports)) { + return MaybeLocal(); + } + + Local defaultName = ArgConverter::ConvertToV8String(isolate, "default"); + if (module->SetSyntheticModuleExport(isolate, defaultName, exports).IsNothing()) { + return MaybeLocal(); + } + + Local keys; + if (!ExportKeys(context, exports).ToLocal(&keys)) { + return MaybeLocal(); + } + for (uint32_t i = 0; i < keys->Length(); i++) { + Local key; + Local value; + if (!keys->Get(context, i).ToLocal(&key) || !key->IsString()) { + return MaybeLocal(); + } + if (key.As()->StringEquals(defaultName)) { + continue; + } + if (!exports->Get(context, key).ToLocal(&value) || + module->SetSyntheticModuleExport(isolate, key.As(), value).IsNothing()) { + return MaybeLocal(); + } + } + + Local resolver; + if (!Promise::Resolver::New(context).ToLocal(&resolver) || + !resolver->Resolve(context, Undefined(isolate)).FromMaybe(false)) { + return MaybeLocal(); + } + return resolver->GetPromise(); +} + +} // namespace + +bool NsBuiltinModules::IsBuiltinScheme(const std::string& specifier) { + return HasPrefix(specifier, kNsPrefix) || HasPrefix(specifier, kNodePrefix); +} + +bool NsBuiltinModules::IsNsScheme(const std::string& specifier) { + return HasPrefix(specifier, kNsPrefix); +} + +bool NsBuiltinModules::IsRegistered(const std::string& specifier) { + return Find(specifier) != nullptr; +} + +std::string NsBuiltinModules::NotFoundMessage(const std::string& specifier) { + return "No such built-in module: " + specifier; +} + +MaybeLocal NsBuiltinModules::GetExports(Local context, + const std::string& specifier) { + const Registration* registration = Find(specifier); + if (registration == nullptr) { + return MaybeLocal(); + } + + Isolate* isolate = v8::Isolate::GetCurrent(); + RealmState& realm = GetRealm(isolate); + auto it = realm.exports.find(specifier); + if (it == realm.exports.end()) { + if (!Instantiate(context, *registration)) { + return MaybeLocal(); + } + it = realm.exports.find(specifier); + if (it == realm.exports.end()) { + return MaybeLocal(); + } + } + return it->second->Get(isolate); +} + +MaybeLocal NsBuiltinModules::GetModule(Local context, + const std::string& specifier) { + Isolate* isolate = v8::Isolate::GetCurrent(); + RealmState& realm = GetRealm(isolate); + + auto it = realm.modules.find(specifier); + if (it != realm.modules.end()) { + Local cached = it->second->Get(isolate); + if (!cached.IsEmpty() && cached->GetStatus() != Module::kErrored) { + return cached; + } + delete it->second; + realm.modules.erase(it); + } + + Local exports; + if (!GetExports(context, specifier).ToLocal(&exports)) { + return MaybeLocal(); + } + + Local defaultName = ArgConverter::ConvertToV8String(isolate, "default"); + std::vector> exportNames{defaultName}; + Local keys; + if (!ExportKeys(context, exports).ToLocal(&keys)) { + return MaybeLocal(); + } + for (uint32_t i = 0; i < keys->Length(); i++) { + Local key; + if (!keys->Get(context, i).ToLocal(&key) || !key->IsString()) { + return MaybeLocal(); + } + if (!key.As()->StringEquals(defaultName)) { + exportNames.push_back(key.As()); + } + } + + Local module = Module::CreateSyntheticModule( + isolate, ArgConverter::ConvertToV8String(isolate, specifier), + MemorySpan>(exportNames.data(), exportNames.size()), + PopulateSyntheticModule); + // No dependencies and no user code, so the module can be driven to its + // final state here; importers then only ever see an evaluated module. + if (!module->InstantiateModule(context, &NoDependencies).FromMaybe(false)) { + return MaybeLocal(); + } + if (module->Evaluate(context).IsEmpty()) { + return MaybeLocal(); + } + + realm.modules[specifier] = new Persistent(isolate, module); + return module; +} + +Local NsBuiltinModules::GetFormatFunc(Local context) { + Isolate* isolate = v8::Isolate::GetCurrent(); + RealmState& realm = GetRealm(isolate); + if (realm.format != nullptr) { + return realm.format->Get(isolate); + } + if (realm.formatUnavailable) { + return Local(); + } + + TryCatch tc(isolate); + Local exports; + Local format; + if (!GetExports(context, "ns:util").ToLocal(&exports) || + !exports->Get(context, ArgConverter::ConvertToV8String(isolate, "format")).ToLocal(&format) || + !format->IsFunction()) { + // The caller degrades silently, so this is the only place the failure + // is ever visible. + __android_log_write(ANDROID_LOG_WARN, "JS", + "Warning: console failed to initialize the ns:util formatter"); + // One attempt per realm: a broken builtin must not make every log call + // recompile it. + realm.formatUnavailable = true; + return Local(); + } + + realm.format = new Persistent(isolate, format.As()); + return format.As(); +} + +void NsBuiltinModules::onDisposeIsolate(Isolate* isolate) { + std::lock_guard lock(realmsMutex); + auto it = isolateToRealm.find(isolate); + if (it != isolateToRealm.end()) { + delete it->second; + isolateToRealm.erase(it); + } +} + +} // namespace tns diff --git a/test-app/runtime/src/main/cpp/NsBuiltinModules.h b/test-app/runtime/src/main/cpp/NsBuiltinModules.h new file mode 100644 index 000000000..4fb41639e --- /dev/null +++ b/test-app/runtime/src/main/cpp/NsBuiltinModules.h @@ -0,0 +1,66 @@ +#ifndef NSBUILTINMODULES_H_ +#define NSBUILTINMODULES_H_ + +#include + +#include "v8.h" + +namespace tns { + +/* + * The `ns:` builtin modules and their `node:` compatibility shims, as + * specified by docs/ns-builtin-modules.md. Every module is a per-realm + * singleton built from a RuntimeBuiltins source, so a specifier always yields + * the same frozen exports object and the same synthetic ES module. + */ +class NsBuiltinModules { +public: + /* + * Any `ns:`/`node:` specifier, registered or not. The CommonJS resolver + * claims all of them so a prefixed name can never reach the filesystem; + * bare specifiers (`util`) keep resolving through npm untouched. + */ + static bool IsBuiltinScheme(const std::string& specifier); + + /* + * The `ns:` half of the scheme, whose unknown names must fail rather than + * fall through to any legacy handling. + */ + static bool IsNsScheme(const std::string& specifier); + + // Whether a module of that name exists. + static bool IsRegistered(const std::string& specifier); + + /* + * Frozen exports object of a registered specifier. Empty when the + * specifier is not registered (nothing thrown) or when the module failed + * to build (an exception is pending). + */ + static v8::MaybeLocal GetExports(v8::Local context, + const std::string& specifier); + + /* + * Evaluated synthetic module exporting the same values as GetExports under + * their own names plus `default` (the exports object itself). + */ + static v8::MaybeLocal GetModule(v8::Local context, + const std::string& specifier); + + /* + * What every resolver reports for a specifier in the scheme that names no + * module. + */ + static std::string NotFoundMessage(const std::string& specifier); + + /* + * `ns:util`'s format, used by console.* for %-substitution. Empty when the + * module could not be built; callers degrade instead of failing the log. + */ + static v8::Local GetFormatFunc(v8::Local context); + + static void onDisposeIsolate(v8::Isolate* isolate); +}; + +} // namespace tns + +#endif /* NSBUILTINMODULES_H_ */ diff --git a/test-app/runtime/src/main/cpp/console/Console.cpp b/test-app/runtime/src/main/cpp/console/Console.cpp index f260e36ab..b0a5ca213 100644 --- a/test-app/runtime/src/main/cpp/console/Console.cpp +++ b/test-app/runtime/src/main/cpp/console/Console.cpp @@ -18,6 +18,7 @@ #include "ArgConverter.h" #include "BuiltinLoader.h" #include "Console.h" +#include "NsBuiltinModules.h" #include "robin_hood.h" namespace tns { @@ -81,6 +82,12 @@ static void getNativeWrapperHintCallback(const v8::FunctionCallbackInfo context) { auto isolate = v8::Isolate::GetCurrent(); + if (!getInspectFunction(isolate).IsEmpty()) { + // inspect.js installs a non-configurable global.__inspect, so a second + // run in the same realm would throw. + return; + } + v8::Local hintFunc; if (!v8::Function::New(context, getNativeWrapperHintCallback).ToLocal(&hintFunc)) { __android_log_write(ANDROID_LOG_WARN, LOG_TAG, @@ -109,6 +116,11 @@ void Console::initInspect(v8::Local context) { isolateToInspect.emplace(isolate, new v8::Persistent(isolate, result.As())); } +v8::Local Console::getInspect(v8::Local context) { + initInspect(context); + return getInspectFunction(v8::Isolate::GetCurrent()); +} + // depth < 0 leaves the builtin's own default (2); console.dir passes 4. static std::string inspectValue(v8::Isolate* isolate, const v8::Local& val, int depth = -1) { auto context = isolate->GetCurrentContext(); @@ -212,9 +224,32 @@ std::string buildStringFromArg(v8::Isolate* isolate, const v8::Local& std::string buildLogString(const v8::FunctionCallbackInfo& info, int startingIndex = 0) { auto isolate = info.GetIsolate(); v8::HandleScope scope(isolate); + auto context = isolate->GetCurrentContext(); + auto argLen = info.Length(); + + // console.* follows Node: the arguments go through util.format, so the + // first one may carry %-substitutions and the rest are appended + // space-separated. + auto format = argLen > startingIndex ? NsBuiltinModules::GetFormatFunc(context) + : v8::Local(); + if (!format.IsEmpty()) { + std::vector> formatArgs; + formatArgs.reserve(argLen - startingIndex); + for (int i = startingIndex; i < argLen; i++) { + formatArgs.push_back(info[i]); + } + v8::TryCatch tc(isolate); + v8::Local result; + if (format->Call(context, v8::Undefined(isolate), static_cast(formatArgs.size()), + formatArgs.data()).ToLocal(&result) && + result->IsString()) { + return ArgConverter::ConvertToString(result.As()); + } + } + + // ns:util unavailable or the formatter threw: per-argument rendering. std::stringstream ss; - auto argLen = info.Length(); if (argLen) { for (int i = startingIndex; i < argLen; i++) { // separate args with a space diff --git a/test-app/runtime/src/main/cpp/console/Console.h b/test-app/runtime/src/main/cpp/console/Console.h index 8cd160e3b..afe715b3e 100644 --- a/test-app/runtime/src/main/cpp/console/Console.h +++ b/test-app/runtime/src/main/cpp/console/Console.h @@ -33,6 +33,10 @@ class Console { static void onDisposeIsolate(v8::Isolate* isolate); + // Builds this realm's inspect function if it isn't there yet. Public + // so ns:util can re-export the same instance. + static v8::Local getInspect(v8::Local context); + private: using ConsoleAPIType = v8_inspector::ConsoleAPIType; diff --git a/test-app/runtime/src/main/cpp/js/README.md b/test-app/runtime/src/main/cpp/js/README.md index 7025849a5..e0599e6fb 100644 --- a/test-app/runtime/src/main/cpp/js/README.md +++ b/test-app/runtime/src/main/cpp/js/README.md @@ -9,17 +9,24 @@ build time a CMake custom command runs `tools/js2c.mjs`, which embeds them into ## Contract (Node's module wrapper + internalBinding idiom) Every file is compiled as a **function body** via `v8::ScriptCompiler::CompileFunction` -with the fixed parameters `exports`, `module`, `binding` and `primordials`: +with the fixed parameters `exports`, `require`, `module`, `binding` and +`primordials`: ```js const { someNative, anotherNative } = binding; const { ArrayPrototypeSlice, ObjectCreate } = primordials; +const { inspect } = require("ns:util"); module.exports = somethingTheCallSiteNeeds; ``` - `binding` is a plain object of natives built by the C++ call site; a file that needs nothing from C++ simply doesn't mention it. +- `require` resolves **builtin specifiers only** (`ns:util`, `node:util`, …), + never a path or a package; an unknown one throws + `No such built-in module: `. It is how a `node:` shim consumes the + `ns:` module it adapts, and it materializes that module on first use. + Requiring a module that is still loading throws rather than recursing. - `primordials` is the frozen intrinsics snapshot built by `primordials.js` (see below), the same object for every builtin in an isolate. - **`module.exports` is the export channel** — whatever it holds when the file @@ -35,6 +42,10 @@ module.exports = somethingTheCallSiteNeeds; internal `__inspect` global): budgeted output, no getter invocation, tamper-immune via primordials. Console routes all object formatting through it. +- `ns-util.js` is the `ns:util` module app code requires and `node-util.js` the + `node:util` shim: one source file per specifier, the shim owning every bit of + Node compatibility. See `docs/ns-builtin-modules.md` for the cross-runtime + contract. - Destructure `binding` and `primordials` once, at the top of the file, so the file's dependencies are visible and greppable. @@ -46,8 +57,8 @@ module.exports = somethingTheCallSiteNeeds; (`URLSearchParams`, …) capture it into a file-level `const`. - No `import`/`export` — these are classic function bodies, not modules. - ESLint (`eslint.config.mjs` at the repo root, `npm run lint`) declares - `exports`, `module`, `binding`, `primordials` and the reachable native - globals; `no-undef` is the typo net. If a builtin starts using a new native + `exports`, `require`, `module`, `binding`, `primordials` and the reachable + native globals; `no-undef` is the typo net. If a builtin starts using a new native global, add it there. `no-restricted-properties` fails the lint on direct use of the captured statics (`JSON.stringify`, `Object.defineProperty`, …). Uncurried instance methods can't be matched that way, so `list.slice()` diff --git a/test-app/runtime/src/main/cpp/js/node-util.js b/test-app/runtime/src/main/cpp/js/node-util.js new file mode 100644 index 000000000..a89c2826e --- /dev/null +++ b/test-app/runtime/src/main/cpp/js/node-util.js @@ -0,0 +1,16 @@ +"use strict"; + +// The `node:util` compatibility shim: a documented subset of Node's util, +// backed by `ns:util` (docs/ns-builtin-modules.md). Compiled the first time +// `node:util` is resolved, so an app that never touches the `node:` scheme +// never pays for it. +// +// Every adaptation to Node's API belongs in this file — argument shapes, +// option names, aliases, anything that has to track Node rather than the +// runtime. The `ns:` modules stay free of compatibility knowledge, so the two +// surfaces can diverge without either one carrying the other's baggage. + +const { ObjectFreeze } = primordials; +const { inspect, format } = require("ns:util"); + +module.exports = ObjectFreeze({ inspect, format }); diff --git a/test-app/runtime/src/main/cpp/js/ns-util.js b/test-app/runtime/src/main/cpp/js/ns-util.js new file mode 100644 index 000000000..a48fdf5b2 --- /dev/null +++ b/test-app/runtime/src/main/cpp/js/ns-util.js @@ -0,0 +1,151 @@ +"use strict"; + +// The `ns:util` builtin module. See docs/ns-builtin-modules.md for the +// cross-runtime contract. `inspect` is created once per realm by +// Console::getInspect and handed in here, so the module and console share one +// formatter instance. + +const { inspect } = binding; +const { + JSONStringify, + Number, + NumberParseFloat, + NumberParseInt, + ObjectFreeze, + ObjectIs, + String, + StringPrototypeCharCodeAt, + StringPrototypeSlice, +} = primordials; + +const CHAR_PERCENT = 37; +const CHAR_UPPERCASE_O = 79; +const CHAR_LOWERCASE_D = 100; +const CHAR_LOWERCASE_F = 102; +const CHAR_LOWERCASE_I = 105; +const CHAR_LOWERCASE_J = 106; +const CHAR_LOWERCASE_O = 111; +const CHAR_LOWERCASE_S = 115; + +function formatNumber(value) { + return ObjectIs(value, -0) ? "-0" : String(value); +} + +// %d and %i keep a bigint's "n" suffix (%f, which parses a float, does not). +// Symbols refuse numeric conversion, so they render as NaN rather than throw. +function formatInteger(value, toNumber) { + if (typeof value === "symbol") { + return "NaN"; + } + if (typeof value === "bigint") { + return String(value) + "n"; + } + return formatNumber(toNumber(value)); +} + +function stringify(value) { + try { + return JSONStringify(value); + } catch (ignored) { + // Circular structures are the common case; anything else JSON refuses + // (a throwing toJSON) is not worth taking the log call down for. + return "[Circular]"; + } +} + +// A value spliced in for %s: strings go in raw, everything else through +// inspect (which already honors custom toString overrides). +function toDisplayString(value) { + return typeof value === "string" ? value : inspect(value); +} + +function format(...args) { + const first = args[0]; + let str = ""; + let a = 0; + let join = ""; + + if (typeof first === "string") { + if (args.length === 1) { + return first; + } + let lastPos = 0; + // A trailing "%" cannot start a substitution, hence length - 1. + for (let i = 0; i < first.length - 1; i++) { + if (StringPrototypeCharCodeAt(first, i) !== CHAR_PERCENT) { + continue; + } + const nextChar = StringPrototypeCharCodeAt(first, ++i); + if (a + 1 === args.length) { + // Out of arguments: only %% still means something. + if (nextChar === CHAR_PERCENT) { + str += StringPrototypeSlice(first, lastPos, i); + lastPos = i + 1; + } + continue; + } + let replacement; + switch (nextChar) { + case CHAR_LOWERCASE_S: + replacement = toDisplayString(args[++a]); + break; + case CHAR_LOWERCASE_D: { + const value = args[++a]; + replacement = formatInteger(value, Number); + break; + } + case CHAR_LOWERCASE_I: { + const value = args[++a]; + replacement = formatInteger(value, NumberParseInt); + break; + } + case CHAR_LOWERCASE_F: { + const value = args[++a]; + replacement = typeof value === "symbol" ? "NaN" : formatNumber(NumberParseFloat(value)); + break; + } + case CHAR_LOWERCASE_J: + replacement = stringify(args[++a]); + break; + case CHAR_LOWERCASE_O: + case CHAR_UPPERCASE_O: + replacement = inspect(args[++a]); + break; + case CHAR_PERCENT: + str += StringPrototypeSlice(first, lastPos, i); + lastPos = i + 1; + continue; + default: + // Not a placeholder: "100%" and friends survive verbatim. + continue; + } + if (lastPos !== i - 1) { + str += StringPrototypeSlice(first, lastPos, i - 1); + } + str += replacement; + lastPos = i + 1; + } + if (lastPos !== 0) { + // The format string consumed argument 0 and set the separator for the + // extras that follow it. + a++; + join = " "; + if (lastPos < first.length) { + str += StringPrototypeSlice(first, lastPos); + } + } + } + + // Leftovers (or every argument, when there was no format string) are + // appended space-separated. + while (a < args.length) { + const value = args[a]; + str += join + toDisplayString(value); + join = " "; + a++; + } + + return str; +} + +module.exports = ObjectFreeze({ inspect, format }); diff --git a/test-app/runtime/src/main/cpp/js/primordials.js b/test-app/runtime/src/main/cpp/js/primordials.js index 7685ec7f4..119bb4187 100644 --- a/test-app/runtime/src/main/cpp/js/primordials.js +++ b/test-app/runtime/src/main/cpp/js/primordials.js @@ -23,6 +23,7 @@ const intrinsics = { // Constructors. Date, Map, + Number, Proxy, Set, String, @@ -35,8 +36,11 @@ const intrinsics = { ArrayBufferIsView: ArrayBuffer.isView, ArrayIsArray: Array.isArray, JSONStringify: JSON.stringify, + NumberParseFloat: Number.parseFloat, + NumberParseInt: Number.parseInt, ObjectCreate: Object.create, ObjectDefineProperty: Object.defineProperty, + ObjectFreeze: Object.freeze, ObjectGetOwnPropertyDescriptor: Object.getOwnPropertyDescriptor, ObjectGetOwnPropertySymbols: Object.getOwnPropertySymbols, ObjectGetPrototypeOf: Object.getPrototypeOf, @@ -69,6 +73,7 @@ const intrinsics = { SetPrototypeDelete: uncurryThis(Set.prototype.delete), SetPrototypeHas: uncurryThis(Set.prototype.has), SetPrototypeValues: uncurryThis(Set.prototype.values), + StringPrototypeCharCodeAt: uncurryThis(String.prototype.charCodeAt), StringPrototypeIndexOf: uncurryThis(String.prototype.indexOf), StringPrototypeSlice: uncurryThis(String.prototype.slice), SymbolPrototypeToString: uncurryThis(Symbol.prototype.toString), diff --git a/tools/js2c.mjs b/tools/js2c.mjs index 987b3bc31..12a654c5c 100644 --- a/tools/js2c.mjs +++ b/tools/js2c.mjs @@ -76,6 +76,7 @@ for (const file of inputs) { } if (!file.endsWith('.js')) { console.error(`error: builtins must be .js function bodies, got: ${file}`); + console.error('(.mjs/source-text modules are deliberately unsupported — see docs/ns-builtin-modules.md)'); process.exit(1); } const stem = basename(file).replace(/\.js$/, '');