Skip to content

feat(sdk,plugins): a failed action shows the API's error body, callTool takes { throwOnError: false }, and a failed workflow is an error result - #1095

Merged
frontegg-david merged 6 commits into
fix/edge-managed-e2efrom
feat/skilled-error-bodies
Nov 4, 2026
Merged

frontegg-david merged 6 commits into
fix/edge-managed-e2efrom
feat/skilled-error-bodies

Conversation

@frontegg-david

Copy link
Copy Markdown
Contributor

What and why

When a run_workflow call failed, the model couldn't see why, and a script couldn't react:

  • A 4xx or 5xx answer reached the script as action "getInvoice" failed with status 404. The API's error body, such as {"error":"refund_too_large","message":"Refund 500 exceeds the paid amount 120.","maxRefund":120}, was dropped. The result envelope already kept it; the run_workflow bridge dropped it.
  • One failed call ended the whole script, and AgentScript has no try/catch.
  • run_workflow answered { success: false, error } without isError. MCP clients saw a successful tool call, and the generated CLI's run-workflow exited 0.

This PR makes failures something the model and scripts can act on.

The error carries the API's error body. A failed action's error is now action "<id>" failed with status <N>: <body>.

  • A JSON body is compacted with JSON.stringify. A text body is kept as it is. An empty body adds nothing.
  • The body is cut at 2048 characters (MAX_ERROR_BODY_LENGTH), and a cut ends with ….
  • The body goes through redactJson from @frontmcp/utils, the redaction the SDK's call recorder uses. It removes Bearer/Basic credentials, JWTs and URL credentials, plus the field the operation's credential travels in (authorization, or an apiKey binding's header or query name). The plugin had no header/token redaction of its own to reuse for this.
  • New option outbound.errorBodies: 'include' | 'status-only', default 'include'. It's validated by the Zod options schema. 'status-only' keeps the status-only message, for APIs whose error bodies hold internal details.
  • Failures with no answer keep their own messages: authority denied, input validation, timeout, abort, a redirect, an oversized response, an output-schema mismatch.

callTool(actionId, input, { throwOnError: false }) returns { ok, status, data } instead of throwing. The default still throws.

  • On success: ok: true, the status and the data.
  • On an HTTP failure: ok: false, the status, and the redacted, capped body as data. A body over the cap comes as its cut text. With 'status-only' there's no data.
  • On a failure with no answer (authority, input validation, timeout, abort, an unknown action): ok: false, status: 0 and an error string.
  • Audit records don't change: a failure is still audited as a failure.
  • How the options reach the handler: @enclave-vm/core/worker's InterpreterAdapter (2.17.0, the latest) calls the tool handler with two arguments only (toolHandler(name, args)), so a third callTool argument never reaches it. The tool now runs a short callTool(actionId, input, options) function ahead of the script. It folds the options into the input under a reserved key, __callToolOptions, and the handler strips that key before validation. The script is transformed with additionalIdentifiers: ['callTool'], the way the full Enclave keeps its own globals, so script calls reach that function. A failed call still throws from the host, since the interpreter has no throw statement. Once the interpreter passes a third argument, this function can go.

A failed workflow is an error result. When success is false, run_workflow answers a CallToolResult with isError: true.

  • The result keeps the same structuredContent (success, error, stats), and its text content is that JSON, as for a success.
  • The output schema already describes both shapes.
  • MCP clients now see a failed call, and the generated CLI's run-workflow exits 1, through the CLI's existing isError handling.
  • The tools/call flow already sent a CallToolResult a tool returns as it is. But @Tool's return-type check refused that return type on a tool with an outputSchema. The check now accepts Out | CallToolResult. It's a type-only change in libs/sdk, with a type test.

Description. run_workflow's description now covers throwOnError, the error body and the error result. It's shorter: 1008 characters, within the 1024-character tool-description budget (it was 1182).

This PR is stacked on #1094 (Cloudflare managed mode), which is stacked on #1093.

User-facing change

  • What changed:
    • A failed action's error in run_workflow includes the API's error body: redacted, JSON compacted, cut at 2 KB with ….
    • New option outbound.errorBodies: 'include' | 'status-only' (default 'include').
    • callTool(actionId, input, { throwOnError: false }) returns { ok, status, data, error? }.
    • A failed workflow is an isError: true result with the same structuredContent.
    • The generated CLI's run-workflow exits 1 when the workflow fails.
    • @Tool accepts a CallToolResult return type on a tool with an outputSchema.
  • Before → after:
    • action "refundInvoice" failed with status 422 → action "refundInvoice" failed with status 422: {"error":"refund_too_large","message":"Refund 500 exceeds the paid amount 120.","maxRefund":120}.
    • callTool(id, input, { throwOnError: false }) ignored its third argument and threw → it returns { ok: false, status: 422, data: {…} }.
    • A failed run_workflow result had no isError and the CLI exited 0 → it has isError: true and the CLI exits 1.
    • @Tool({ outputSchema }) with execute(): Promise<Out | CallToolResult> failed to compile → it compiles.
  • frontmcp.dev pages affected (if known):
  • Skills catalog entries updated:
    • create-tool/references/error-handling.md: returning a CallToolResult with isError and structured content.
    • frontmcp-deployment/references/build-for-cli.md: run-workflow exit codes and a troubleshooting row.
    • frontmcp-deployment/references/deploy-to-cloudflare-skills-only.md: error bodies, throwOnError, isError, and two common mistakes.
    • The plugin README also covers all of this.

Tests

Added:

  • action-outcome.spec.ts, 15 tests:
    • success, with and without data;
    • a failure with no answer;
    • JSON and text bodies, over the cap, exactly at the cap, empty;
    • 'status-only';
    • redaction of an echoed apiKey field, a Bearer value and a JWT;
    • the failure messages.
  • error-bodies.integration.spec.ts, 11 tests, with a real server and the real @enclave-vm interpreter:
    • JSON, text and over-the-cap bodies, and an echoed credential redacted;
    • 'status-only', thrown and returned;
    • throwOnError: false on success, on an HTTP failure (a script that branches and retries), and on failures with no answer (input validation, authority);
    • the audit records of a returned failure;
    • isError on a failed workflow and not on a successful one.
  • run-workflow.tool.spec.ts: 10 new tests and 8 updated:
    • the description budget;
    • the prelude and the transform options;
    • the error result and its absence on success;
    • the bridge's body, 'status-only' and throwOnError paths.
  • index.spec.ts: outbound.errorBodies defaults to 'include', accepts 'status-only' and rejects anything else.
  • SDK tool-type-safety.spec.ts: a tool with an outputSchema returning Out | CallToolResult typechecks.
  • demo-e2e-skilled-openapi (execution.e2e.spec.ts), over MCP with the production settings:
    • the mock's refund endpoint answers 422 with a JSON body when the refund exceeds the paid amount, and that body reaches the model's error text;
    • a throwOnError: false script branches on a 404 and returns a fallback;
    • a failed workflow has isError: true with the same structured content, and a successful one doesn't;
    • the 404 test now expects the body.
  • demo-e2e-cli-exec (cli-skilled-openapi.e2e.spec.ts): run-workflow exits 0 for a workflow that succeeds and 1 for one that fails, with isError and the error in --output json.

Run, after merging #1094 and rebuilding sdk, adapters, plugin-skilled-openapi, testing and cli:

  • plugin suite: 40 suites, 586 tests passed. ssrf-guard.spec.ts failed once in four full runs under load: its real DNS lookup of a .invalid host is slow. It passes alone and on the other runs, and this PR doesn't change it.
  • nx test demo-e2e-skilled-openapi: 5 suites, 52 tests passed.
  • demo-e2e-cli-exec:
    • cli-skilled-openapi.e2e.spec.ts: 11 passed;
    • cli-errors.e2e.spec.ts: 10 passed;
    • cli-login-authorities.e2e.spec.ts: 4 skipped, as always on macOS.
  • bundle-size e2e: 12 suites, 63 tests passed. The SDK change is type-only, so the budgets are unchanged.
  • SDK decorator tests: 11 suites, 68 passed. libs/skills: 3 suites, 249 passed.
  • nx run <project>:typecheck passed for plugin-skilled-openapi, sdk, demo-e2e-skilled-openapi and demo-e2e-cli-exec (and its typecheck-e2e). tsc --noEmit on the plugin's and the SDK's tsconfig.lib.json passed.
  • prettier --check and eslint (0 errors) passed on the changed files. fix-unused-imports.mjs changed nothing.

…chema

The tools/call flow already sends a CallToolResult a tool returns as it is,
whatever its outputSchema. The decorator's return-type check refused one on a
tool with an outputSchema, so such a tool could not report a failure as an
isError result that keeps its structured content without a cast.
…y, and callTool takes { throwOnError: false }

A 4xx/5xx answer reached the script as `action "x" failed with status 404`,
so the model could not see what the API objected to, and one failed call
ended the script, which has no try/catch. Now:

- the thrown error ends with the API's error body: JSON compacted, text as
  it is, cut at 2 KB with an ellipsis, and run through redactJson with the
  field the operation's credential travels in;
- outbound.errorBodies: 'status-only' keeps the status-only message;
- callTool(actionId, input, { throwOnError: false }) returns
  { ok, status, data } (status 0 and an error for a failure with no answer).
  The interpreter passes its tool handler two arguments, so a callTool
  declared ahead of the program folds the options into the input under a
  reserved key;
- a failed workflow is an isError result with the same structured content,
  so MCP clients see a failed call and the generated CLI exits 1.

The run_workflow description says so, within the 1024-character budget.
… on a 404, and a failed run-workflow exits 1

Over MCP against the production-settings billing API, whose refund endpoint
now answers 422 with a JSON body when a refund exceeds the paid amount; and
on a CLI built with the plugin, run-workflow exits 0 for a workflow that
succeeds and 1 for one that fails.
… try/catch in AgentScript

The CLI page says what a non-zero run-workflow exit means, and the Cloudflare
skills-only page points a script that wants try/catch at throwOnError: false.
@frontegg-david
frontegg-david marked this pull request as ready for review November 4, 2026 19:31
@frontegg-david

Copy link
Copy Markdown
Contributor Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

Your organization has reached its usage spending cap. Adjust your spending cap in the billing tab.

Next included review available in 27 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 2 included reviews currently available.

Learn how review limits work.

Review configuration:

Run configuration
Commits

Reviewing files that changed from the base of the PR and between 97e78d1 and 51d1cb4.

Files selected for processing: 24
  • apps/e2e/demo-e2e-cli-exec/e2e/cli-skilled-openapi.e2e.spec.ts
  • apps/e2e/demo-e2e-skilled-openapi/e2e/execution.e2e.spec.ts
  • apps/e2e/demo-e2e-skilled-openapi/e2e/helpers/mock-billing-api.ts
  • apps/e2e/demo-e2e-skilled-openapi/e2e/helpers/skilled-client.ts
  • libs/sdk/src/common/decorators/__tests__/tool-type-safety.spec.ts
  • libs/sdk/src/common/decorators/tool.decorator.ts
  • libs/skills/catalog/create-tool/references/error-handling.md
  • libs/skills/catalog/frontmcp-deployment/references/build-for-cli.md
  • libs/skills/catalog/frontmcp-deployment/references/deploy-to-cloudflare-skills-only.md
  • plugins/plugin-skilled-openapi/README.md
  • plugins/plugin-skilled-openapi/src/__tests__/audit-and-startup.spec.ts
  • plugins/plugin-skilled-openapi/src/__tests__/error-bodies.integration.spec.ts
  • plugins/plugin-skilled-openapi/src/__tests__/index.spec.ts
  • plugins/plugin-skilled-openapi/src/__tests__/openapi-runtime.spec.ts
  • plugins/plugin-skilled-openapi/src/__tests__/path-parameters.spec.ts
  • plugins/plugin-skilled-openapi/src/__tests__/pinned-fetch.spec.ts
  • plugins/plugin-skilled-openapi/src/__tests__/run-workflow.tool.spec.ts
  • plugins/plugin-skilled-openapi/src/__tests__/ssrf-guard.spec.ts
  • plugins/plugin-skilled-openapi/src/enclave-peer.d.ts
  • plugins/plugin-skilled-openapi/src/executor/__tests__/action-outcome.spec.ts
  • plugins/plugin-skilled-openapi/src/executor/action-outcome.ts
  • plugins/plugin-skilled-openapi/src/skilled-openapi.types.ts
  • plugins/plugin-skilled-openapi/src/tools/run-workflow.schema.ts
  • plugins/plugin-skilled-openapi/src/tools/run-workflow.tool.ts
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

@coderabbitai

coderabbitai Bot commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor
Action not completed

Review rate limited.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@github-actions

github-actions Bot commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

Performance Test Results

Status: ✅ All tests passed

Summary

Project Tests Passed Warnings Failed Leaks
✅ demo-e2e-agents 4 4 0 0 0
✅ demo-e2e-cache 11 11 0 0 0
✅ demo-e2e-codecall 4 4 0 0 0
✅ demo-e2e-config 4 4 0 0 0
✅ demo-e2e-direct 3 3 0 0 0
✅ demo-e2e-elicitation 1 1 0 0 0
✅ demo-e2e-errors 4 4 0 0 0
✅ demo-e2e-hooks 3 3 0 0 0
✅ demo-e2e-multiapp 4 4 0 0 0
✅ demo-e2e-notifications 3 3 0 0 0
✅ demo-e2e-openapi 2 2 0 0 0
✅ demo-e2e-providers 4 4 0 0 0
✅ demo-e2e-public 4 4 0 0 0
✅ demo-e2e-redis 15 15 0 0 0
✅ demo-e2e-remember 4 4 0 0 0
✅ demo-e2e-remote 5 5 0 0 0
✅ demo-e2e-serverless 2 2 0 0 0
✅ demo-e2e-skills 15 15 0 0 0
✅ demo-e2e-standalone 2 2 0 0 0
✅ demo-e2e-transport-recreation 3 3 0 0 0
✅ demo-e2e-ui 4 4 0 0 0

Total: 101 tests across 21 projects

📊 View full report in workflow run


Generated at: 2026-11-04T20:04:57.572Z
Commit: 2517d2f1

@frontegg-david
frontegg-david merged commit fe574e1 into release/1.11.x Nov 4, 2026
46 checks passed
@frontegg-david
frontegg-david deleted the feat/skilled-error-bodies branch November 4, 2026 23:23
frontegg-david added a commit that referenced this pull request Nov 4, 2026
… body, callTool takes { throwOnError: false }, and a failed workflow is an error result (#1095)

Cherry-picked from #1095 (merged to release/1.11.x)
Original commit: fe574e1
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants