Repository navigation
Conversation
07b2aaf to
88ab4fa
Compare
|
Preview build of b5019a2: |
|
The graph walk that this PR lets Bun remove is also a release regression. Since Bun 1.4.0, // A: import { v } from "B"; export const a = v + 1;
// B: import "A"; export const v = 1;
await A.link(s => ({ A, B })[s]);
await A.evaluate();
// Bun 1.4.0 to main: ReferenceError: Cannot access 'v' before initialization.
// Node v26.3.0, Bun 1.3.14, and the same two modules as files: ok
Tests for the Bun PR are on the branch Two of the guards matter for where the evaluation steps run. With Three more results from Node v26.3.0 that the Bun side has to keep:
One more point, on the cost this PR lists for every import edge to a builtin, JSON or CommonJS record (one load and one branch in |
31ba0c6 to
f5362a4
Compare
…from the module record module.evaluate() walked the graph itself. It called evaluate() on every dependency in request order, each with its own Evaluate() and its own promise, and then on the module. A dependency that threw after a top-level await rejected a promise that no caller could reach, so the process ended with an unhandled rejection. A cycle ran in the wrong order, and a SyntheticModule ran before the source text modules imported ahead of it. evaluate() now calls Evaluate() of the module's record once. JSC runs the whole graph in the order of the specification: - SyntheticModuleRecord::evaluate() gives the record's owner to Bun__runSyntheticModuleEvaluationSteps(), so an evaluateCallback runs where the walk reaches its module, in the async context of the caller. - import.meta is created when the module evaluates it for the first time. initializeImportMeta runs then, through the realm's moduleLoaderCreateImportMetaProperties hook. The module's NodeVMScriptFetcher keeps the callback, like importModuleDynamically. - A module that imports a module with a top-level await runs in the async context that evaluate() was called in. The status and the error of a SourceTextModule come from its record. A record is made in the realm of the module's context, so its jobs go to the queue of that context. A timeout or a SIGINT that stops the walk, an evaluateCallback, or a job of an afterEvaluate context gives the modules it stopped the error of the run that was cut short. The JavaScriptCore changes are in oven-sh/WebKit#796. WEBKIT_VERSION points at the preview build of that pull request until it merges.
…from the module record module.evaluate() walked the graph itself. It called evaluate() on every dependency in request order, each with its own Evaluate() and its own promise, and then on the module. A dependency that threw after a top-level await rejected a promise that no caller could reach, so the process ended with an unhandled rejection. A cycle ran in the wrong order, and a SyntheticModule ran before the source text modules imported ahead of it. evaluate() now calls Evaluate() of the module's record once. JSC runs the whole graph in the order of the specification: - SyntheticModuleRecord::evaluate() gives the record's owner to Bun__runSyntheticModuleEvaluationSteps(), so an evaluateCallback runs where the walk reaches its module, in the async context of the caller. - import.meta is created when the module evaluates it for the first time. initializeImportMeta runs then, through the realm's moduleLoaderCreateImportMetaProperties hook. The module's NodeVMScriptFetcher keeps the callback, like importModuleDynamically. - A module that imports a module with a top-level await runs in the async context that evaluate() was called in. The status and the error of a SourceTextModule come from its record. A record is made in the realm of the module's context, so its jobs go to the queue of that context. A timeout or a SIGINT that stops the walk, an evaluateCallback, or a job of an afterEvaluate context gives the modules it stopped the error of the run that was cut short. The JavaScriptCore changes are in oven-sh/WebKit#796. WEBKIT_VERSION points at the preview build of that pull request until it merges.
…from the module record module.evaluate() walked the graph itself. It called evaluate() on every dependency in request order, each with its own Evaluate() and its own promise, and then on the module. A dependency that threw after a top-level await rejected a promise that no caller could reach, so the process ended with an unhandled rejection. A cycle ran in the wrong order, and a SyntheticModule ran before the source text modules imported ahead of it. evaluate() now calls Evaluate() of the module's record once. JSC runs the whole graph in the order of the specification: - SyntheticModuleRecord::evaluate() gives the record's owner to Bun__runSyntheticModuleEvaluationSteps(), so an evaluateCallback runs where the walk reaches its module, in the async context of the caller. - import.meta is created when the module evaluates it for the first time. initializeImportMeta runs then, through the realm's moduleLoaderCreateImportMetaProperties hook. The module's NodeVMScriptFetcher keeps the callback, like importModuleDynamically. - A module that imports a module with a top-level await runs in the async context that evaluate() was called in. The status and the error of a SourceTextModule come from its record. A record is made in the realm of the module's context, so its jobs go to the queue of that context. A timeout or a SIGINT that stops the walk, an evaluateCallback, or a job of an afterEvaluate context gives the modules it stopped the error of the run that was cut short. The JavaScriptCore changes are in oven-sh/WebKit#796. WEBKIT_VERSION points at the preview build of that pull request until it merges.
A Synthetic Module Record has [[EvaluationSteps]]: what the host runs when the record is evaluated. SyntheticModuleRecord::evaluate() had no such step, so a host with steps (Bun's node:vm SyntheticModule) had to walk the module graph itself, ahead of Evaluate(), to run them. A record can now have an owner. The first evaluate() gives the owner to Bun__runSyntheticModuleEvaluationSteps(), which the host defines, and the record forgets it: the steps run once. That happens where InnerModuleEvaluation reaches the record, so the steps run at their depth-first position, and what they throw fails the importers the way a module body that throws does. The owner is kept in the generator frame field every module record has. A synthetic record has no bytecode, so it never used that field. The record does not grow and has nothing more to visit. jsc has no host: a weak definition of the function that does nothing is what it links. No record has an owner there. A record without an owner pays 6 instructions per import edge (64 to 70 in the jsc shell, x86-64): one load, two compares, two branches that are not taken, one move.
f5362a4 to
b5019a2
Compare
…from the module record module.evaluate() walked the graph itself. It called evaluate() on every dependency in request order, each with its own Evaluate() and its own promise, and then on the module. A dependency that threw after a top-level await rejected a promise that no caller could reach, so the process ended with an unhandled rejection. A cycle ran in the wrong order, and a SyntheticModule ran before the source text modules imported ahead of it. evaluate() now calls Evaluate() of the module's record once. JSC runs the whole graph in the order of the specification: - SyntheticModuleRecord::evaluate() gives the record's owner to Bun__runSyntheticModuleEvaluationSteps(), so an evaluateCallback runs where the walk reaches its module, in the async context of the caller. - import.meta is created when the module evaluates it for the first time. initializeImportMeta runs then, through the realm's moduleLoaderCreateImportMetaProperties hook. The module's NodeVMScriptFetcher keeps the callback, like importModuleDynamically. - A module that imports a module with a top-level await runs in the async context that evaluate() was called in. The status and the error of a SourceTextModule come from its record. A record is made in the realm of the module's context, so its jobs go to the queue of that context. A timeout or a SIGINT that stops the walk, an evaluateCallback, or a job of an afterEvaluate context gives the modules it stopped the error of the run that was cut short. The JavaScriptCore changes are in oven-sh/WebKit#796. WEBKIT_VERSION points at the preview build of that pull request until it merges.
|
Closing: oven-sh/bun#46004 no longer needs this hook. |
Problem
[[EvaluationSteps]]of the host.SyntheticModuleRecord::evaluate()runs none.node:vmtherefore walks the graph itself, ahead ofEvaluate(), to run avm.SyntheticModulecallback. Since Bun 1.4.0 that walk makes a legalvm.SourceTextModuleimport cycle throw a TDZReferenceError.Fix
setEvaluationStepsOwner). The firstevaluate()gives the owner toBun__runSyntheticModuleEvaluationSteps(), which the host defines, and the record forgets it.InnerModuleEvaluationreaches the record, so the steps run at their depth-first position and a throw fails the importers.JSTests/modules.yaml(1650 runs) with this branch'sjsc.jsccannot make a record with an owner, so the hook's tests are in the Bun pull request that uses it (link follows).Background
InnerModuleEvaluationwalks a module graph depth first. For a synthetic record it callsSyntheticModuleRecord::evaluate(), once per import edge.Downsides
jscshell). In Bun that is each import of a builtin module (63 to 69).USE(BUN_JSC_ADDITIONS), not tested in this repository.Notes
This PR had four commits. It has one now. The other three are not needed for the regression:
AsyncModuleExecutionDoneis the second commit of JSC: evaluate dynamically imported modules under the importer's async context #274.import.metacreated at its first evaluation changes bytecode for every ES module. It is on the branchrobobun/f48be7fb/import-meta-first-evaluationand needs its own decision and numbers for each tier.JSModuleRecord::isSuspendedAtTopLevelAwait()is gone. Bun readsField::Stateitself, as it already does inZigGlobalObject.cpp.Why a link-time hook and an internal field. One host function for every record costs no bytes per record and no setup per VM. It follows
Bun__analyzeTranspiledModuleinJSModuleLoader.cpp: JSC has a weak definition that does nothing, forjsc, and the host's definition replaces it at link.sizeof(SyntheticModuleRecord)stays 312 and its visit is unchanged. The readers ofField::StateandField::Frameare the bytecode of a module body,Interpreter::executeModuleProgramandJSModuleRecord::isTopLevelExecutionFinished. None of them runs for a synthetic record. JSC does not look at the owner: the host's function checks its type.The steps run once.
evaluate()runs once per import edge and once per read of a deferred namespace. The record forgets the owner before it calls the host.evaluate()always returnsundefined: a promise would makeevaluateModuleSync()callevaluate()again.The cost. A record without an owner pays one load, two compares, two branches that are not taken and one move. The counts are gdb single-step counts from the entry of
AbstractModuleRecord::innerModuleEvaluationto its return, for a JSON module in thejscshell of thelinux-amd64-ltolane and fornode:fsin a release build of Bun.#653 and this PR. ec68522 gives
SyntheticSourceProvideran evaluator, keeps it in the record as aSharedTask, remembers what it throws, and rethrows that to every later importer. This PR differs in two ways, both because ofvm.SyntheticModule:Stronghandle, and record -> task -> module -> record is then a cycle the GC cannot collect. An owner in an internal field is visited like any other reference.SyntheticModulewhose callback threw earlier. So nothing is remembered here.They agree on the rest: the steps run once, they are taken before the call, and a termination is not remembered. One entry point could serve both: the owner in the frame field, the steps as a task or a function, and a flag that says whether the error is remembered (it fits in the state field, which a synthetic record does not use either). This PR does not build that, because it changes #653's side too.