[frontegg-david]

Stacked on #1006

What and why

Proposal item 4.2: offload CPU-heavy work so the server keeps answering.

A FrontMCP server answers every request on one event loop. A tool or resource that parses a large file, walks a
repository or computes for seconds blocks it, and every other call waits, ping included. There was no way to run
such work off the event loop short of hand-rolling worker_threads, which also meant no budget, no cancellation, no
shutdown and no guarantee about what reaches the worker.

What this PR changes:

• this.offload(moduleRef, args, { budgetMs, signal }) on ExecutionContextBase, so tools, resources, prompts,
  agents, jobs and skills all have it. It runs an exported function of another module in a worker thread of the scope's
  pool and resolves with what it returns. moduleRef is { module, export? }: module is an absolute path, a
  file: URL string or a URL (new URL('./x.js', import.meta.url)); a relative path, a package name or a
  non-file: URL is refused with InvalidOffloadCallError, and so are non-array args, a missing export and a
  non-positive budgetMs. The type parameter types the arguments and the result
  (this.offload<typeof countWords>(ref, [text, 3])).
• @Resource({ worker: true | { budgetMs } }), and the same on @ResourceTemplate, resource() and
  resourceTemplate() (validated by Zod in the metadata schemas). The read-resource flow runs as before; only its
  execute stage hands execute(uri, params) to the pool instead of calling it, so authorities, enabled, every hook,
  output validation and the answer budget still run on the main thread through the flow. The decorator records the
  module it was evaluated in (from V8 call sites, so a compiled build names its .js file even with source maps on, and
  a module of an installed package is named itself rather than the module that required the package); the worker
  imports that module, finds the resource's reader in a registry its own copy of the SDK filled while the module
  loaded (keyed by module, name and URI or URI template, so two resources of one module or bundle that share a name stay
  apart), builds the resource there and calls execute(). A bundle that inlines @frontmcp/sdk (a SEA binary, mcpb,
  lambda, vercel) has no separate module to import, and its worker reads are refused with
  InvalidOffloadCallError saying so; the --target node bundle keeps the SDK external and works.
• The pool (offload/offload-pool.ts): workers start on demand, are reused, are capped at maxWorkers (calls past
  it queue) and stop after idleTimeoutMs idle (an idle worker is unref'd, so it never holds the process open). Each
  worker runs one call at a time.
  • Budget: counts from the moment the function starts in its worker (the worker reports it), so a fresh worker
    loading the SDK does not eat a small budget; loading is bounded by the larger of the call's budget and
    offload.budgetMs. Past it the worker is terminated, the call rejects with OffloadBudgetExceededError
    (OFFLOAD_BUDGET_EXCEEDED, public, 504) and the server warns, once per offloaded function (module and export, or
    resource); later overruns of the same one are logged at verbose level.
  • Abort: the call's signal (default this.signal) terminates the worker, running or not yet started, and the
    call rejects with OffloadAbortedError (OFFLOAD_ABORTED). this.signal moves from ToolContext/JobContext up
    to ExecutionContextBase, and a resource read now gets the request's signal (the MCP handler's extra.signal), so a
    tool's timeout and a client's cancel both stop the worker.
  • Crashes: an uncaught error, process.exit() or an out-of-memory exit in the worker rejects the call with
    OffloadWorkerError and the pool starts a new worker for the next call; so does a worker that cannot start.
  • Errors from the function reach the caller with their name, message, stack and code. A public FrontMCP error
    (an McpError, recognised by its class brand, with isPublic) is rebuilt as a PublicMcpError with its code,
    status, remedy and JSON-RPC code and data, so a worker resource's ResourceNotFoundError is answered with -32002
    and data.uri, as an inline read is. Any other error stays internal, as it is on the main thread, even one with an
    isPublic property, so its message is not sent to a client in production.
• Node only: OffloadService (below) checks getRuntimeContext().runtime first and throws
  OffloadUnsupportedRuntimeError naming the runtime (edge, browser, bun, deno). Nothing imports
  worker_threads at module load: the pool module is loaded with import() on the first call and requires
  node:worker_threads, node:os and node:url inside it. The browser bundle gains about 1.5 KB (the option schema,
  two error classes and the service); the read flow loads its worker helper with import() only for a worker
  resource, and a new worker offload entry in demo-e2e-bundle-size keeps the pool, the worker source, the flow
  helper and the worker-side reader out of the bundle.
• Shutdown through M01's lifecycle: OffloadService is a GLOBAL provider in every scope's default providers
  (useValue), and its onShutdown() closes the pool, so every path that disposes a scope (graceful HTTP and stdio
  shutdown, create()/createDirect() dispose(), the last connect() client) terminates the workers; the calls
  still running or queued reject with OffloadAbortedError ("the server is shutting down"), and later calls are refused.
• New option @FrontMcp({ offload: { maxWorkers, idleTimeoutMs, budgetMs, env } }), a strict Zod schema; an
  invalid value fails @FrontMcp with InvalidDecoratorMetadataError naming offload.

What crosses the worker boundary (also in the catalog and the JSDoc):

• In: the module URL and export name (or, for a resource, the module, its name, the URI and its parameters) and args,
  by structured clone. The worker's process.env holds only the variables named in offload.env (default
  ['NODE_ENV']): the pool passes env to new Worker() and the worker deletes every other variable before it imports
  anything, which also drops what --env-file loaded (an inherited --env-file flag re-reads the file in a worker
  even with env given; found while testing). The worker inherits the server's execArgv (loaders, conditions), never
  its argv; a preload among those flags (--require, --import) runs in each worker before the worker clears its
  environment and sees an --env-file's variables (passing a filtered execArgv instead is refused by Node for V8
  flags such as --max-old-space-size, so this is documented rather than changed). No DI container, scope, session, auth info, access token or request headers cross: a worker resource is
  built with authInfo: {} and a provider registry whose every method throws ProviderNotAvailableError naming the
  resource, so this.get(), this.scope, this.auth and this.context throw there, this.tryGet() returns
  undefined, and this.fetch() is a plain fetch.
• Out: the return value by structured clone (OffloadNotCloneableError for a function, a symbol and the like, in
  either direction); an error's name, message, stack, code, whether it is a public FrontMCP error and, if so, its
  statusCode, remedy and JSON-RPC payload; a worker resource's log lines, which the server writes with
  resource:<name>.
• A worker's stdout is written to the server's stderr (forwarded chunk by chunk, not piped, so eleven or more workers do
  not trip a MaxListenersExceededWarning on process.stderr), so a console.log in offloaded code never corrupts a
  stdio transport.
• Only a function the module exports is called: an export name the exports only inherit (toString, a function's
  call) is refused like a missing one.

Two runtime details found while building it:

• A module imported in a worker may import the server's own entry (a resource declared in the @FrontMcp file, or a
  bundle). The worker marks its thread (Symbol.for('@frontmcp/sdk:offload-worker')) before it imports anything, and
  @FrontMcp then records its metadata and serves nothing, as in schema-extract mode.
• tsx registers its hooks on the main thread only (its loader checks isMainThread), so under frontmcp dev or
  node --import tsx a worker could not load a decorated .ts resource. When the server's execArgv names tsx, the
  pool passes the paths of tsx's CommonJS and ES module registration APIs and the worker registers both before it
  imports anything. A tsx without those APIs is skipped, so JavaScript modules still run.

Deviations from the brief / PLAN.md:

• args is the array of the function's arguments (spread into the call), typed from the function by the type
  parameter.
• "Warn once per entry" is keyed by the offloaded function (module and export, or the worker resource); the warning is
  written with the calling context's logger, so it carries the tool's or resource's name.
• worker also takes { budgetMs }, since a server-wide budget does not fit every resource.
• The pool is per scope (a splitByApp app has its own), so N split apps run up to N × maxWorkers threads.

Code layout: offload/ holds the service (offload.service.ts, always loaded, tiny), the pool and the worker source
(offload-pool.ts, offload-worker.source.ts, loaded on demand), the worker-resource registry the decorators call
(worker-resources.ts, no runtime imports of the context classes, so the per-module browser build has no evaluation
cycle), the reader a worker loads on demand (worker-resource.reader.ts) and the flow helper the read flow loads on
demand (read-in-worker.ts).
captureCallerFile() joins captureCallerDir() in common/utils/caller-dir.utils.ts.

User-facing change

• What changed:
  • New method this.offload(moduleRef, args, options?) on every execution context, and the types
    OffloadModuleRef, OffloadCallOptions and OffloadFunction exported from @frontmcp/sdk.
  • New resource option worker?: boolean | { budgetMs?: number } on @Resource, @ResourceTemplate, resource() and
    resourceTemplate() (type ResourceWorkerOption). Messages: worker.budgetMs must be a whole number of milliseconds, ... must be greater than 0.
  • New server option offload: { maxWorkers?, idleTimeoutMs?, budgetMs?, env? } (defaults: available parallelism
    minus one, between 1 and 4; 30000; 30000; ['NODE_ENV']), strict. Messages: offload.maxWorkers must be at least 1, offload.<field> must be a whole number of milliseconds, ... must be greater than 0, offload.env must be a list of environment variable names; an invalid value throws InvalidDecoratorMetadataError for offload.
  • New errors: OffloadBudgetExceededError (OFFLOAD_BUDGET_EXCEEDED, public, 504: Offloaded "<name>" ran past its <n> ms budget and was stopped), OffloadAbortedError (OFFLOAD_ABORTED, public, 499: ... was stopped: its call was cancelled / the server is shutting down), OffloadUnsupportedRuntimeError (OFFLOAD_UNSUPPORTED_RUNTIME),
    InvalidOffloadCallError (INVALID_OFFLOAD_CALL), OffloadNotCloneableError (OFFLOAD_NOT_CLONEABLE),
    OffloadWorkerError (OFFLOAD_WORKER_FAILED).
  • New log line (warning, once per offloaded function): Offloaded "<name>" (<module>#<export>) ran past its <n> ms budget; its worker was terminated. Later overruns of it are logged at verbose level.
  • this.signal is now declared on ExecutionContextBase. In a resource read it was always undefined and is now
    the request's signal, aborted when the client cancels the read or the connection closes; a resource's
    this.fetch() is unchanged (it does not pass the signal). Prompts, agents and skills still get none.
  • A context subclass that already had a method or property named offload or signal of another type now clashes
    with the base class (TypeScript reports it): rename it.
• Before → after:
  • CPU-heavy work in a tool or resource blocked every other call → this.offload() or worker: true runs it in a
    worker thread, and ping and other calls keep answering (the e2e measures it).
  • Unchanged: a resource without worker reads on the main thread as before; nothing starts a worker until the first
    call that needs one; tool and job this.signal behave as before.
  • A public FrontMCP error thrown in a worker is answered as it would be inline (same JSON-RPC code, data and
    remedy); any other error thrown there is internal, as inline.
• frontmcp.dev pages affected (if known): the tool and resource references (/reference/sdk/tool,
  /reference/sdk/resource: this.offload(), worker, this.signal on resources), the @FrontMcp configuration
  reference (offload), the errors reference (the six new errors), and the guides on performance and on running under
  stdio (a worker's console output goes to stderr).
• Skills catalog entries updated:
  • frontmcp-development: new references/offload-cpu-work.md (when to offload, this.offload(), module references,
    budget and abort, worker: true and what the worker can and can't reach, what crosses the boundary, the offload
    option, errors, patterns, checklist, troubleshooting; also which errors stay public across the boundary, installed
    packages and bundles that inline the SDK, preloads in the server's flags, N × maxWorkers threads with
    splitByApp, a tsx without registration APIs), references/create-resource.md (worker, this.offload(),
    this.signal), references/decorators-guide.md (offload? and worker? rows), SKILL.md (routing row, list,
    related skills), skills-manifest.json (the new reference).
  • create-tool: references/execution-context.md (this.offload() row).

Tests

• Unit (libs/sdk: 45 new tests in offload/__tests__, 6 added to existing specs):
  • offload/__tests__/offload-pool.spec.ts (27, real worker threads and modules written to a temp dir): result and
    thread, file: URL / URL / CommonJS modules, reuse, the maxWorkers cap and queueing, idle timeout, budget
    overrun (termination, typed error, warn once then verbose, the pool recovers), the budget counted from the start of
    the function (a module that takes 400 ms to load, a 400 ms call, a 600 ms budget), the pool default budget, abort of
    a running call, of an already-aborted signal and of a queued call, non-cloneable arguments and results, the
    function's error, a public FrontMCP error keeping its JSON-RPC code, data and remedy (toSdkMcpError answers
    -32002 with data.uri, and -32003 with the remedy), an error that only claims isPublic staying internal, an
    export name the exports only inherit refused, a worker's console output forwarded to stderr without a pipe per
    worker, a tsx without registration APIs skipped, worker exit and crash, a worker that cannot start, every refused module reference, the environment
    (NODE_ENV only by default, offload.env names), a worker resource read with log forwarding, an unregistered
    worker resource, close() and tsxApiOf().
  • offload/__tests__/offload.service.spec.ts (5): refused on edge, browser and bun naming the runtime without
    starting a pool, one pool for every call and closed on shutdown, refused after shutdown.
  • offload/__tests__/offload.context.spec.ts (5, through create()): a tool's this.offload(), a tool timeout
    terminating its worker (a 1-worker pool serves the next call), dispose() rejecting the call in flight with
    OffloadAbortedError, the read-resource flow handing a worker resource and template to the pool with module,
    name, URI, params and budget, and a budget error answered by the read.
  • offload/__tests__/worker-resources.spec.ts (8): the recorded module and key, the reader with no DI container,
    respond(), fail(), this.get() / this.scope refused, log forwarding, two resources of one module that share a
    name read apart, a function-style resource, a resource whose module is unknown, the worker marker.
  • common/utils/__tests__/caller-dir.utils.spec.ts (+4, one for a module of an installed package being named
    itself), common/decorators/__tests__/front-mcp-stdio.decorator.spec.ts
    (+2: serves nothing in a worker, an invalid offload refused with InvalidDecoratorMetadataError).
  • Coverage of the new code: 97.6% lines, 96.4% statements, 91% branches.
  • The whole SDK suite: 593 suites, 7738 tests passed (--maxWorkers=50%).
• E2E:
  • demo-e2e-local-runtime → worker-offload.e2e.spec.ts (new; the server runs under tsx over HTTP with
    offload.maxWorkers: 1): while a worker: true read keeps its worker busy for 1.5 s, the longest ping stays under
    a third of that (measured: 16 ms at most over 49 pings), while the same work read inline holds a ping for more than
    half of it (measured: 1509 ms), and the worker read reports isMainThread: false; a read past its 300 ms budget is
    refused with the budget message, after which ping, the heavy read (on a new worker, as the only one was
    terminated) and a tool's this.offload() all answer. Bounds are relative to the busy time, so a loaded machine does
    not fail them. A ResourceNotFoundError thrown in a worker template is answered with -32002, its message and
    data.uri, as inline (it was -32602 before the review fix). The project now declares its sdk:build dependency,
    and its jest config maps @frontmcp/testing. demo-e2e-local-runtime: 4 suites, 7 tests passed.
  • demo-e2e-dts-consumer → offload-must-compile.ts (new): this.offload<typeof fn>() typing its arguments and
    result (and a must-fail for swapped arguments), worker on @Resource / @ResourceTemplate (a must-fail for
    worker: 'yes'), the offload option (a must-fail for budgetMs: '15s'). 18 tests passed.
  • demo-e2e-bundle-size: new worker offload feature entry (the pool, the worker source, the flow helper and the
    worker-side reader stay out); 10 tests passed (create() + one tool: 1,313,024 bytes, budget 1,315,000).
  • demo-e2e-cloudflare: 2 suites, 16 tests passed (1 suite skipped, as on the base).
• nx test skills: 249 passed. Lint of the changed files: 0 errors (only warnings on lines this PR does not touch).
  Prettier: clean. fix-unused-imports: nothing to remove. Types: nx build sdk (declarations) and a --noEmit
  compile of the new spec files.

[frontegg-david]

@coderabbitai review

[coderabbitai]

Warning

Review limit reached

This review is too large to run within your organization's remaining usage spending cap. Raise or remove your spending cap in the billing tab, then retry.

Next included review available in 32 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 2 included reviews currently available. Your 51 included PR review attempts over the past 7 days set your current allowance at 2 reviews per hour.

Learn how review limits work.

Review configuration:

Run configuration

• Source mode: Auto
• Repository: agentfront/frontmcp
• Branch: feat/m11-worker-offload
• File: .coderabbit.yaml
• Commit: 966f070
• Layers: Repository YAML
• Review profile: CHILL
• Plan: Essentials
• Run ID: f728371b-33c3-4b68-ac27-f4b20482ab55

Commits

Reviewing files that changed from the base of the PR and between f0f46db and 966f070.

Files selected for processing: 52

• apps/e2e/demo-e2e-bundle-size/e2e/browser-bundle-size.e2e.spec.ts
• apps/e2e/demo-e2e-dts-consumer/e2e/dts-consumer.e2e.spec.ts
• apps/e2e/demo-e2e-dts-consumer/fixture/offload-must-compile.ts
• apps/e2e/demo-e2e-dts-consumer/fixture/tsconfig.json
• apps/e2e/demo-e2e-local-runtime/e2e/worker-offload.e2e.spec.ts
• apps/e2e/demo-e2e-local-runtime/fixture/offload/crunch.tool.ts
• apps/e2e/demo-e2e-local-runtime/fixture/offload/main.ts
• apps/e2e/demo-e2e-local-runtime/fixture/offload/reports.resource.ts
• apps/e2e/demo-e2e-local-runtime/fixture/offload/spin.ts
• apps/e2e/demo-e2e-local-runtime/jest.e2e.config.ts
• apps/e2e/demo-e2e-local-runtime/project.json
• libs/sdk/src/common/decorators/__tests__/front-mcp-stdio.decorator.spec.ts
• libs/sdk/src/common/decorators/front-mcp.decorator.ts
• libs/sdk/src/common/decorators/resource.decorator.ts
• libs/sdk/src/common/interfaces/execution-context.interface.ts
• libs/sdk/src/common/interfaces/job.interface.ts
• libs/sdk/src/common/interfaces/resource.interface.ts
• libs/sdk/src/common/interfaces/tool.interface.ts
• libs/sdk/src/common/metadata/front-mcp.metadata.ts
• libs/sdk/src/common/metadata/resource.metadata.ts
• libs/sdk/src/common/tokens/front-mcp.tokens.ts
• libs/sdk/src/common/tokens/resource.tokens.ts
• libs/sdk/src/common/types/options/index.ts
• libs/sdk/src/common/types/options/offload/index.ts
• libs/sdk/src/common/types/options/offload/schema.ts
• libs/sdk/src/common/utils/__tests__/caller-dir.utils.spec.ts
• libs/sdk/src/common/utils/caller-dir.utils.ts
• libs/sdk/src/errors/index.ts
• libs/sdk/src/errors/offload.errors.ts
• libs/sdk/src/index.ts
• libs/sdk/src/offload/__tests__/offload-pool.spec.ts
• libs/sdk/src/offload/__tests__/offload.context.spec.ts
• libs/sdk/src/offload/__tests__/offload.service.spec.ts
• libs/sdk/src/offload/__tests__/worker-resources.spec.ts
• libs/sdk/src/offload/index.ts
• libs/sdk/src/offload/offload-pool.ts
• libs/sdk/src/offload/offload-worker.globals.ts
• libs/sdk/src/offload/offload-worker.source.ts
• libs/sdk/src/offload/offload.service.ts
• libs/sdk/src/offload/offload.types.ts
• libs/sdk/src/offload/read-in-worker.ts
• libs/sdk/src/offload/worker-resource.reader.ts
• libs/sdk/src/offload/worker-resources.ts
• libs/sdk/src/resource/flows/read-resource.flow.ts
• libs/sdk/src/resource/resource.instance.ts
• libs/sdk/src/scope/scope.instance.ts
• libs/skills/catalog/create-tool/references/execution-context.md
• libs/skills/catalog/frontmcp-development/SKILL.md
• libs/skills/catalog/frontmcp-development/references/create-resource.md
• libs/skills/catalog/frontmcp-development/references/decorators-guide.md
• libs/skills/catalog/frontmcp-development/references/offload-cpu-work.md
• libs/skills/catalog/skills-manifest.json

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

[coderabbitai]

Action not completed

Review rate limited.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

[github-actions]

Performance Test Results

Status: ✅ All tests passed

Summary

Project	Tests	Passed	Warnings	Failed	Leaks
✅ demo-e2e-agents	4	4	0	0	0
✅ demo-e2e-cache	11	11	0	0	0
✅ demo-e2e-codecall	4	4	0	0	0
✅ demo-e2e-config	4	4	0	0	0
✅ demo-e2e-direct	3	3	0	0	0
✅ demo-e2e-elicitation	1	1	0	0	0
✅ demo-e2e-errors	4	4	0	0	0
✅ demo-e2e-hooks	3	3	0	0	0
✅ demo-e2e-multiapp	4	4	0	0	0
✅ demo-e2e-notifications	3	3	0	0	0
✅ demo-e2e-openapi	2	2	0	0	0
✅ demo-e2e-providers	4	4	0	0	0
✅ demo-e2e-public	4	4	0	0	0
✅ demo-e2e-redis	15	15	0	0	0
✅ demo-e2e-remember	4	4	0	0	0
✅ demo-e2e-remote	5	5	0	0	0
✅ demo-e2e-serverless	2	2	0	0	0
✅ demo-e2e-skills	15	15	0	0	0
✅ demo-e2e-standalone	2	2	0	0	0
✅ demo-e2e-transport-recreation	3	3	0	0	0
✅ demo-e2e-ui	4	4	0	0	0

Total: 101 tests across 21 projects

📊 View full report in workflow run

---

Generated at: 2026-11-02T16:52:59.210Z
Commit: 579a7436