Skip to content

Commit 1b99d7c

Browse files
authored
sea: mount the SEA VFS in workers at layer 0
A single executable application built with "useVfs" mounted the VFS serving its "assets" in the main thread only, at a mount point numbered in mount order, so a worker started from a script among the assets could not load it. On Windows this failed as an invalid package.json at \.\nul\package.json, because paths under the NUL device open it. "useVfs" is the --vfs-load of a single executable application: its "assets" or its "vfsArchive" archive are the source, and its "main" is the entry point. Mount it at layer 0, the mount point reserved for the --vfs-load source, whichever provider serves it, and mount it in workers too, as was already done for an archive. Other mounts are numbered from 1, as everywhere else. To keep layer 0 free, reject --vfs-load in every single executable application rather than only in one built with "vfsArchive": its entry point is fixed, and resolving modules against its files is what "useVfs" already does. --build-sea refuses it in "execArgv", and the executable at startup when it comes from --node-options or from the execArgv of a worker. A worker started from a path into a VFS that is not mounted in it, such as one mounted with vfs.mount() in its parent, now fails with ERR_VFS_NOT_MOUNTED. Fixes: #67223 Assisted-by: a coding agent Signed-off-by: Manjeet Singh <024manjeetsingh@gmail.com> PR-URL: #67239 Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Philipp Dunkel <pip@pipobscure.com>
1 parent dc6b820 commit 1b99d7c

25 files changed

Lines changed: 273 additions & 80 deletions

‎doc/api/cli.md‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -4004,9 +4004,9 @@ comes from anywhere else, such as the real file system, needs nothing added.
40044004
the command line's decision, and the environment must not be able to redirect
40054005
it.
40064006

4007-
A [single executable application][] built with `"vfsArchive"` mounts that
4008-
archive at the mount point reserved for `--vfs-load`, so `--vfs-load` cannot be
4009-
used in it.
4007+
`--vfs-load` cannot be used in a [single executable application][]: its entry
4008+
point is the `"main"` it was built with, and with `"useVfs"` its own virtual
4009+
file system is mounted at the mount point reserved for `--vfs-load`.
40104010

40114011
```console
40124012
$ node --vfs-load=app.zip

‎doc/api/errors.md‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3598,6 +3598,17 @@ entry types are found.
35983598
A `--vfs-load` source does not exist, is neither a regular file nor a
35993599
directory, or is a source no provider claims.
36003600

3601+
<a id="ERR_VFS_NOT_MOUNTED"></a>
3602+
3603+
### `ERR_VFS_NOT_MOUNTED`
3604+
3605+
A path into a virtual file system was used in a thread in which that virtual
3606+
file system is not mounted. Each thread has its own mounts: a virtual file
3607+
system mounted with [`vfs.mount()`][] is not mounted in the [`Worker`][]
3608+
threads, so a worker cannot be started from a script in it. The source
3609+
mounted with [`--vfs-load`][] and the virtual file system of a
3610+
[single executable application][] are mounted in every thread.
3611+
36013612
<a id="ERR_VM_DYNAMIC_IMPORT_CALLBACK_MISSING"></a>
36023613

36033614
### `ERR_VM_DYNAMIC_IMPORT_CALLBACK_MISSING`
@@ -4787,6 +4798,7 @@ An error occurred trying to allocate memory. This should never happen.
47874798
[`--force-fips`]: cli.md#--force-fips
47884799
[`--no-addons`]: cli.md#--no-addons
47894800
[`--unhandled-rejections`]: cli.md#--unhandled-rejectionsmode
4801+
[`--vfs-load`]: cli.md#--vfs-loadsource
47904802
[`ABORT_ERR`]: #abort_err
47914803
[`BoundSocket`]: net.md#class-netboundsocket
47924804
[`Class: DOMException`]: globals.md#class-domexception
@@ -4805,6 +4817,7 @@ An error occurred trying to allocate memory. This should never happen.
48054817
[`REPL`]: repl.md
48064818
[`ServerResponse`]: http.md#class-httpserverresponse
48074819
[`Temporal`]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Temporal
4820+
[`Worker`]: worker_threads.md#class-worker
48084821
[`Writable`]: stream.md#class-streamwritable
48094822
[`ZipBuffer`]: zlib.md#class-zlibzipbuffer
48104823
[`ZipFile`]: zlib.md#class-zlibzipfile
@@ -4868,6 +4881,7 @@ An error occurred trying to allocate memory. This should never happen.
48684881
[`util.inspect()`]: util.md#utilinspectobject-options
48694882
[`util.parseArgs()`]: util.md#utilparseargsconfig
48704883
[`v8.startupSnapshot.setDeserializeMainFunction()`]: v8.md#v8startupsnapshotsetdeserializemainfunctioncallback-data
4884+
[`vfs.mount()`]: vfs.md#vfsmount
48714885
[`zlib`]: zlib.md
48724886
[crypto digest algorithm]: crypto.md#cryptogethashes
48734887
[debugger]: debugger.md
@@ -4878,6 +4892,7 @@ An error occurred trying to allocate memory. This should never happen.
48784892
[package map]: packages.md#package-maps
48794893
[relative URL]: https://url.spec.whatwg.org/#relative-url-string
48804894
[self-reference a package using its name]: packages.md#self-referencing-a-package-using-its-name
4895+
[single executable application]: single-executable-applications.md#virtual-file-system-vfs-for-assets
48814896
[special scheme]: https://url.spec.whatwg.org/#special-scheme
48824897
[stream-based]: stream.md
48834898
[syscall]: https://man7.org/linux/man-pages/man2/syscalls.2.html

‎doc/api/single-executable-applications.md‎

Lines changed: 32 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -223,6 +223,13 @@ The VFS supports the `node:fs` operations for reading files and directories.
223223
Since the SEA VFS is read-only, write operations fail with `EROFS`. See the
224224
[VFS documentation][] for the full list of supported operations.
225225

226+
`useVfs` does for a single executable application what [`--vfs-load`][] does
227+
for `node`: the entry point is its `"main"`, and modules resolve against its
228+
own files. `--vfs-load` therefore cannot be used in a single executable
229+
application, whether or not it uses `useVfs`: `--build-sea` rejects it in
230+
`"execArgv"`, and the executable rejects it at startup when it comes from
231+
`--node-options` or from the `execArgv` of a worker.
232+
226233
#### Loading modules from the VFS in a SEA
227234

228235
When `useVfs` is enabled, the main script is executed from inside the
@@ -264,6 +271,26 @@ Module format detection works the same way as on the real file
264271
system: name bundled ES modules with the `.mjs` extension (or provide the
265272
relevant `package.json` files as assets) so they are interpreted as ESM.
266273
274+
#### Worker threads
275+
276+
The virtual file system is mounted at the mount point reserved for
277+
[`--vfs-load`][], which is the same in every thread, and worker threads mount
278+
it as well. A path into it, such as the main script's
279+
`__filename` or a worker script bundled as an asset, can therefore be loaded
280+
in a [`Worker`][], which reaches the other assets the same way the main
281+
script does:
282+
283+
```cjs
284+
const { Worker } = require('node:worker_threads');
285+
const path = require('node:path');
286+
287+
// worker.js is bundled as an asset, at the root of the mount.
288+
new Worker(path.join(__dirname, 'worker.js'));
289+
```
290+
291+
A virtual file system mounted with [`vfs.mount()`][] in one thread is not
292+
mounted in the others, so a worker cannot load a script from it.
293+
267294
#### Serving the assets from a ZIP archive with `"vfsArchive"`
268295
269296
Instead of listing individual `"assets"`, the configuration can point
@@ -299,10 +326,9 @@ executable only embeds the archive; read the files through the file system
299326
APIs instead. `"vfsArchive"` requires `"useVfs": true` and cannot be
300327
combined with `"assets"`.
301328
302-
The archive is mounted at the mount point reserved for [`--vfs-load`][], which
303-
is the same in every thread, and worker threads mount it as well. A path into
304-
the archive, such as the main script's `__filename`, can therefore be loaded
305-
in a worker:
329+
The archive is mounted at the same mount point as the assets would be, and
330+
in [worker threads](#worker-threads) as well. A path into the archive, such
331+
as the main script's `__filename`, can therefore be loaded in a worker:
306332
307333
```cjs
308334
const { Worker, isMainThread } = require('node:worker_threads');
@@ -315,11 +341,6 @@ if (isMainThread) {
315341
}
316342
```
317343
318-
Since that mount point is taken by the archive, `--vfs-load` cannot be used
319-
with `"vfsArchive"`: `--build-sea` rejects it in `"execArgv"`, and the
320-
executable rejects it at startup when it comes from `--node-options` or from
321-
the `execArgv` of a worker.
322-
323344
#### Code cache for the modules in the VFS
324345
325346
When `"useCodeCache": true` is combined with `"useVfs": true`, the code cache
@@ -827,6 +848,7 @@ to help us document them.
827848
[VFS documentation]: vfs.md
828849
[Windows SDK]: https://developer.microsoft.com/en-us/windows/downloads/windows-sdk/
829850
[`--vfs-load`]: cli.md#--vfs-loadsource
851+
[`Worker`]: worker_threads.md#class-worker
830852
[`node:zlib`]: zlib.md
831853
[`process.execPath`]: process.md#processexecpath
832854
[`require()`]: modules.md#requireid
@@ -837,6 +859,7 @@ to help us document them.
837859
[`sea.getRawAsset()`]: #seagetrawassetkey
838860
[`v8.startupSnapshot.setDeserializeMainFunction()`]: v8.md#v8startupsnapshotsetdeserializemainfunctioncallback-data
839861
[`v8.startupSnapshot` API]: v8.md#startup-snapshot-api
862+
[`vfs.mount()`]: vfs.md#vfsmount
840863
[documentation about startup snapshot support in Node.js]: cli.md#--build-snapshot
841864
[fuse]: https://www.electronjs.org/docs/latest/tutorial/fuses
842865
[module loader integration]: vfs.md#module-loader-integration

‎doc/node.1‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -2042,9 +2042,9 @@ comes from anywhere else, such as the real file system, needs nothing added.
20422042
\fB--vfs-load\fR is not permitted in \fBNODE_OPTIONS\fR: which entry point runs is
20432043
the command line's decision, and the environment must not be able to redirect
20442044
it.
2045-
A single executable application built with \fB"vfsArchive"\fR mounts that
2046-
archive at the mount point reserved for \fB--vfs-load\fR, so \fB--vfs-load\fR cannot be
2047-
used in it.
2045+
\fB--vfs-load\fR cannot be used in a single executable application: its entry
2046+
point is the \fB"main"\fR it was built with, and with \fB"useVfs"\fR its own virtual
2047+
file system is mounted at the mount point reserved for \fB--vfs-load\fR.
20482048
.Bd -literal
20492049
$ node --vfs-load=app.zip
20502050
.Ed

‎lib/internal/errors.js‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1964,6 +1964,8 @@ E('ERR_VALID_PERFORMANCE_ENTRY_TYPE',
19641964
'At least one valid performance entry type is required', Error);
19651965
E('ERR_VFS_INVALID_TARGET',
19661966
'%s is not a valid --vfs-load source: must be an existing file or directory', Error);
1967+
E('ERR_VFS_NOT_MOUNTED',
1968+
'%s is in a virtual file system that is not mounted in this thread', Error);
19671969
E('ERR_VM_DYNAMIC_IMPORT_CALLBACK_MISSING',
19681970
'A dynamic import callback was not specified.', TypeError);
19691971
E('ERR_VM_DYNAMIC_IMPORT_CALLBACK_MISSING_FLAG',

‎lib/internal/main/worker_thread.js‎

Lines changed: 10 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,7 @@ const {
1111
ObjectDefineProperty,
1212
PromisePrototypeThen,
1313
RegExpPrototypeExec,
14+
StringPrototypeIncludes,
1415
} = primordials;
1516

1617
const {
@@ -153,9 +154,9 @@ port.on('message', (message) => {
153154
if (getOptionValue('--import').length === 0) {
154155
finishVfsMounts();
155156
}
156-
// Unlike the --vfs-load source, a SEA archive needs no provider that a
157+
// Unlike the --vfs-load source, the SEA VFS needs no provider that a
157158
// preload could register, so its mount is never deferred.
158-
if (getOptionValue('[sea_vfs_archive]')) {
159+
if (getOptionValue('[sea_vfs]')) {
159160
require('internal/vfs/sea').initSeaVfsInWorker();
160161
}
161162
}
@@ -247,6 +248,13 @@ port.on('message', (message) => {
247248
runWebWorkerScript(source, webWorkerData);
248249
break;
249250
}
251+
// A worker does not share the VFS mounts of its parent. Report a path
252+
// into one that is not mounted here instead of letting the real file
253+
// system answer for it. Every such path contains the reserved `vfs`
254+
// directory, which spares other workers from loading the VFS.
255+
if (StringPrototypeIncludes(filename, 'vfs')) {
256+
require('internal/vfs/setup').assertVFSPathMounted(filename);
257+
}
250258
// script filename
251259
// runMain here might be monkey-patched by users in --require.
252260
// XXX: the monkey-patchability here should probably be deprecated.

‎lib/internal/vfs/file_system.js‎

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -55,10 +55,11 @@ const kLayerId = Symbol('kLayerId');
5555
const kReservedRoot = Symbol('kReservedRoot');
5656
const kLoadLayer = Symbol('kLoadLayer');
5757

58-
// Layer 0 is reserved for the file system --vfs-load mounts, so that source is
59-
// at the same mount point in every thread whatever else a thread mounts, and a
60-
// path into it stays valid in a worker. Everything else is numbered from 1, in
61-
// the order it is mounted.
58+
// Layer 0 is reserved for the file system the process boots with: the source
59+
// --vfs-load mounts, or the virtual file system of a single executable
60+
// application, which cannot use --vfs-load. It is then at the same mount point
61+
// in every thread whatever else a thread mounts, and a path into it stays valid
62+
// in a worker. Everything else is numbered from 1, in the order it is mounted.
6263
const kLoadLayerId = 0;
6364
let nextLayerId = kLoadLayerId + 1;
6465

‎lib/internal/vfs/sea.js‎

Lines changed: 9 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -48,37 +48,35 @@ function initSeaVfs(mainCode) {
4848
const { VirtualFileSystem, kLoadLayer } = require('internal/vfs/file_system');
4949

5050
const extraFiles = { __proto__: null, [getSeaMainName()]: mainCode };
51-
const isArchive = isVfsArchiveEnabled();
5251
let provider;
53-
if (isArchive) {
52+
if (isVfsArchiveEnabled()) {
5453
provider = createZipProvider(extraFiles);
5554
} else {
5655
const { SEAProvider } = require('internal/vfs/providers/sea');
5756
provider = new SEAProvider({ extraFiles });
5857
}
5958
// The SEA warning already covers the feature; don't emit the
6059
// VirtualFileSystem experimental warning for the implicit SEA mount.
61-
// An archive is mounted where --vfs-load mounts its source, which is the
62-
// same mount point in every thread, so a worker can reach the code in it.
60+
// "useVfs" is the --vfs-load of a SEA: whichever provider serves it, the
61+
// VFS is mounted where --vfs-load mounts its source, which is the same mount
62+
// point in every thread, so a worker can reach the code in it.
6363
const vfs = new VirtualFileSystem(provider, {
6464
__proto__: null,
6565
emitExperimentalWarning: false,
66-
[kLoadLayer]: isArchive,
66+
[kLoadLayer]: true,
6767
});
6868
vfs.mount();
6969

7070
return vfs;
7171
}
7272

7373
/**
74-
* Mounts the SEA virtual file system in a worker, when it is backed by a
75-
* `"vfsArchive"`, so that a path into the mount in the main thread, such as
76-
* the main script's `__filename`, is also valid in the worker.
74+
* Mounts the SEA virtual file system in a worker, so that a path into the
75+
* mount in the main thread, such as the main script's `__filename`, is also
76+
* valid in the worker.
7777
*/
7878
function initSeaVfsInWorker() {
79-
if (isVfsArchiveEnabled()) {
80-
initSeaVfs(getMainCode());
81-
}
79+
initSeaVfs(getMainCode());
8280
}
8381

8482
/**

‎lib/internal/vfs/sea_compile_cache.js‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -45,7 +45,7 @@ const kTypeScriptExtensions = ['.ts', '.cts', '.mts'];
4545
/**
4646
* Mounts the assets of the SEA configuration the way the executable mounts
4747
* them: the main script at the mount point root, at the layer reserved for
48-
* --vfs-load when they come from an archive (see lib/internal/vfs/sea.js).
48+
* --vfs-load (see lib/internal/vfs/sea.js).
4949
* @param {object} options The SEA configuration, see buildSeaCompileCache()
5050
* @param {string} mainName The name of the main script at the mount root
5151
* @returns {string} The mount point
@@ -70,7 +70,7 @@ function mountAssets(options, mainName) {
7070
const vfs = new VirtualFileSystem(provider, {
7171
__proto__: null,
7272
emitExperimentalWarning: false,
73-
[kLoadLayer]: isArchive,
73+
[kLoadLayer]: true,
7474
});
7575

7676
if (!isArchive) {

‎lib/internal/vfs/setup.js‎

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,7 @@ const {
2222
ERR_INVALID_ARG_TYPE,
2323
ERR_INVALID_STATE,
2424
ERR_MODULE_NOT_FOUND,
25+
ERR_VFS_NOT_MOUNTED,
2526
},
2627
} = require('internal/errors');
2728
const { createENOENT, createEROFS, createEXDEV } = require('internal/vfs/errors');
@@ -176,6 +177,27 @@ function findVFSOrRoot(inputPath) {
176177
return { vfs, normalized };
177178
}
178179

180+
/**
181+
* Throws if `filename` is under the reserved VFS root but no layer mounted in
182+
* this thread owns it. Each thread has its own mounts, so this is the case of
183+
* a path a worker is handed into a VFS its parent mounted. Left to the native
184+
* loader, such a path fails confusingly: on Windows, where the root sits under
185+
* `\\.\nul`, the NUL device is found as the file and as an empty, invalid
186+
* package.json above it.
187+
* @param {string} filename
188+
*/
189+
function assertVFSPathMounted(filename) {
190+
const normalized = normalizeMountedPath(filename);
191+
const root = getNormalizedVfsRoot();
192+
if (normalized !== root && !StringPrototypeStartsWith(normalized, root + sep)) {
193+
return;
194+
}
195+
const layerId = getLayerIdFromPath(normalized);
196+
if (layerId === -1 || !activeVFSLayers.has(layerId)) {
197+
throw new ERR_VFS_NOT_MOUNTED(filename);
198+
}
199+
}
200+
179201
/**
180202
* Resolves a path string to the VFS that serves it for the fs functions,
181203
* or null for a path outside the reserved VFS root. Unlike the loader, the
@@ -1164,6 +1186,7 @@ function uninstallHooks() {
11641186
}
11651187

11661188
module.exports = {
1189+
assertVFSPathMounted,
11671190
registerVFS,
11681191
deregisterVFS,
11691192
};

0 commit comments

Comments
 (0)