Skip to content

fix(app): resolve MCP and HTTP schemas at runtime - #7661

Merged
c121914yu merged 15 commits into
labring:mainfrom
c121914yu:codex/pr-7634-runtime-schema
Sep 7, 2026
Merged

fix(app): resolve MCP and HTTP schemas at runtime#7661
c121914yu merged 15 commits into
labring:mainfrom
c121914yu:codex/pr-7634-runtime-schema

Conversation

@c121914yu

@c121914yu c121914yu commented Sep 4, 2026

Copy link
Copy Markdown
Collaborator

需求背景

MCP/HTTP 的 JSON Schema 属于工具的执行定义。添加工作流节点时,不应把原始 Schema 一起放进节点对象,形成一份重复快照。

本 PR 收敛到“新增节点不携带执行 Schema”,不再扩展到存量数据清理、保存/发布过滤或 debug 响应过滤。运行时保留必要的服务端回源逻辑,确保节点不带 Schema 仍可正确执行。

本 PR 替代 #7634 的实现方案。

Fixes #7477

简化后的范围

保留

  • 添加节点的预览仅携带引用、必要展示信息和转换后的 IO,不携带原始执行 Schema。
  • 普通工作流、工具集展开及 Agent 按 toolId 从当前工具集加载执行定义,并进行参数校验。
  • 保留旧 MCP 结构的读取和执行兼容,不做数据库迁移。
  • 工具集自身的完整定义、编辑和独立调试能力保持可用。

不纳入本 PR

  • 不为创建、保存、发布增加额外的 Schema 过滤层。
  • 不为 debug 的 memoryNodes 增加过滤或新的响应模型。
  • 不进一步拆分 StoreNode/RuntimeNode,也不增加专用 StoreWorkflowNode 存储 Schema。
  • 不清洗存量记录,不增加客户端保存/JSON 导出的清理逻辑。
  • 不修改旧 MCP getChildren 接口的原有响应契约。

因此,本 PR 不是“所有客户端响应都不含 Schema”:工具集编辑、独立调试及原有接口仍可使用完整定义;本次约束的是新添加进工作流的节点对象。

改动方案

1. 列表按已知父项类型分流

列表项已经包含 appType,前端保留该信息,根据 parentId 找到父项类型,不再为了判断类型额外查询详情。

场景 列表接口
系统工具,包括进入系统工具集 POST /api/core/app/tool/getSystemToolTemplates
Agent 根目录、普通目录 POST /api/core/app/list
我的工具根目录、普通工具文件夹 POST /api/core/app/list
我的工具中的 MCP/HTTP 工具集 GET /api/core/app/tool/getToolSetChildren
  • 新增的 getToolSetChildren 只用于已知的 MCP/HTTP 父工具集,返回子工具 ID、名称、说明和图标。
  • Agent 不调用该接口;普通目录也不先进行工具集探测。
  • 搜索、嵌套目录及面包屑返回复用已加载列表项的 appType。
  • parentType 仅供前端选择接口,不传给 list;服务端仍独立鉴权并判断真实资源类型。

2. 添加节点统一通过 getPreviewNode

三个标签点击添加仍统一调用:

GET /api/core/app/tool/getPreviewNode

  • 在服务端将完整 Schema 转成 Node IO。
  • 精确移除执行 Schema 和输入上的 customJsonSchema,不递归删除业务 value/defaultValue 中的同名属性。
  • MCP/HTTP 工具集预览保留引用和名称/说明摘要,不带 URL、请求头、静态请求配置或原始 Schema。
  • 复用已有节点模板 Schema,不再增加重复的预览 Schema 包装。

3. 保留新节点执行所需的回源能力

  • MCP/HTTP 工具定义从当前工具集 App 读取,并复用已有 Schema 编解码逻辑。
  • 普通工具节点、工具集展开和 Agent 不依赖新节点中已移除的 Schema 快照。
  • MCP 使用当前 inputSchema 校验参数;HTTP 执行先对最新定义调用 getHTTPToolRuntimeSchemas,兼容标量、缺失或空 requestSchema 后再校验。普通工作流和 Agent 使用相同逻辑,不回退到节点快照。
  • 鉴权身份保持 runningAppInfo.tmbId,不改为当前调用者。

4. 旧 MCP 只做兼容,不迁移

  • 保留“父工具集 + 独立子 App”和 toolData/system_toolData 的旧执行兜底。
  • 请求头优先识别命名映射,再兼容单个密钥,保留自定义头和已有密文。
  • 将历史空 toolId 视为占位值,在新增预览和运行时回退到有效工具集 ID。
  • 仅在生成新预览时移除已知的 system_toolSetData 隐藏配置槽,保留普通 IO。
  • 预览适配使用独立副本,不回写历史源记录。

验证

  • 简化方案阶段:24 个相关测试文件、644 项局部测试通过
  • 本轮 HTTP 兼容修复:新增 8 个最终执行回归用例,覆盖标量字符串/数字、缺失、空 requestSchema;验证合法请求可以执行,缺少必填、类型错误和不满足约束的参数仍被拒绝。修复前 8 个用例失败,修复后 5 个相关测试文件、199 项测试通过
  • 未运行全仓库测试。
  • TypeScript 检查通过:pnpm --filter @fastgpt/app exec tsc --noEmit --incremental false
  • Prettier、git diff --check 通过;本次简化提交通过提交钩子。
  • 覆盖 Agent/普通目录不探测、MCP/HTTP 按父类型分流、搜索及面包屑返回、预览不含执行 Schema、业务值不误删。
  • 覆盖旧 MCP 父子 App、两种请求头结构、明文/密文、空 ID、工作流与 Agent 执行及源记录不变。
  • HTTP/MCP 工具集独立调试的表单字段、数值转换及未保存参数仍有回归测试。
  • 外部 HTTP/MCP 调用使用 mock,未做真实外部服务联调。

已知检查限制

PR 中 HTTPTools/Edit.tsx 仍存在修改前已有的 react-hooks/set-state-in-effect lint 错误,本次不扩大范围修改。此前完整提交曾因此跳过钩子;本次简化提交未涉及该错误位置。

Copilot AI lite review requested due to automatic review settings September 4, 2026 08:52
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 4, 2026

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review Completed 2026-09-04T08:57:47.730631Z b0b3ce6 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b0b3ce663c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +735 to +737
const toolSetId =
(mcpToolsetVal && 'toolId' in mcpToolsetVal ? mcpToolsetVal.toolId : undefined) ??
String(authApp?._id ?? '');

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Resolve the embedded toolset ID before hydrating children

When an Agent selects an ordinary workflow whose sole node is an MCP or HTTP toolset, authApp is the wrapper workflow and the persisted toolset snapshot has no toolId; this fallback therefore uses the wrapper app's _id. Both getMCPChildren and getHTTPToolList now reject non-toolset app types, so the wrapper expands to zero tools even though its inner node still has the actual toolset ID in pluginId. Use the inner toolset node's pluginId (or preserve it in formatPersonalAppNode) before hydrating either branch.

Useful? React with 👍 / 👎.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

There are confirmed edge cases where legacy HTTP toolset data can fail runtime parsing (dropping baseUrl/customHeaders) and where Copy/Transition storageModules bypass sanitization, potentially reintroducing invalid persisted fields.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

This PR revises how MCP and HTTP tool schemas are handled by stripping schema snapshots from workflow persistence and re-hydrating tool definitions/schemas from the current toolset Apps at runtime (workflow runtime + Agent runtime), addressing MongoDB $-prefixed field storage failures and reducing stale schema snapshots.

Changes:

  • Introduces dedicated “storage boundary” Zod schemas for workflow nodes/tool configs to strip runtime schema fields during persistence/migration.
  • Updates workflow dispatch + Agent tool expansion to authorize toolset Apps at runtime and load MCP/HTTP tool lists + schemas dynamically.
  • Adjusts client preview/toolset UI paths to avoid returning executable schema fields and to support toolset references.
File summaries
File Description
projects/app/src/web/core/app/api/tool.ts Safer access to HTTP toolset toolList for template/tool listing.
projects/app/src/pages/api/core/app/version/publish.ts Persists sanitized workflow nodes via StoreWorkflowNodeItemTypeSchema.
projects/app/src/pages/api/core/app/create.ts Sanitizes stored workflow nodes; encodes toolset schemas for toolset Apps.
projects/app/src/pageComponents/app/detail/WorkflowComponents/Flow/nodes/components/ToolSetList.tsx Guards toolList access for MCP/HTTP toolset config unions.
projects/app/src/pageComponents/app/detail/Edit/HTTPTools/SchemaConfigModal.tsx Handles HTTP toolset config unions when opening schema modal.
projects/app/src/pageComponents/app/detail/Edit/HTTPTools/ManualToolModal.tsx Handles HTTP toolset config unions when reading/updating toolList.
projects/app/src/pageComponents/app/detail/Edit/HTTPTools/Edit.tsx Handles HTTP toolset config unions for edit page state.
packages/service/test/core/workflow/dispatch/utils.test.ts Updates rewriteRuntimeWorkFlow tests for tmbId + runtime toolset resolution.
packages/service/test/core/workflow/dispatch/tools/runTool.test.ts Updates runtime auth + schema loading expectations for MCP/HTTP tools.
packages/service/test/core/workflow/dispatch/ai/agent/sub/tool/utils.test.ts Updates Agent runtime tool expansion to load current toolset schemas.
packages/service/test/core/workflow/dispatch/ai/agent/sub/tool/index.test.ts Updates Agent tool dispatch tests for toolset auth + no version loading.
packages/service/test/core/app/tool/utils/client.test.ts Ensures client preview nodes omit runtime schema fields; legacy MCP hydration coverage.
packages/service/test/core/app/mcp.test.ts Adds type guarding: non-MCP apps should not be treated as MCP toolsets.
packages/service/test/core/app/http.test.ts Adds getHTTPToolList regression test for legacy customHeaders type.
packages/service/core/workflow/dispatch/utils/index.ts Runtime toolset expansion now authorizes toolset Apps and loads tool lists/schemas dynamically.
packages/service/core/workflow/dispatch/index.ts Passes tmbId into rewriteRuntimeWorkFlow for runtime authorization.
packages/service/core/workflow/dispatch/child/runTool.ts Validates HTTP params using latest toolset schema; resolves MCP/HTTP configs from toolset Apps at runtime.
packages/service/core/workflow/dispatch/ai/agent/sub/tool/utils.ts Agent runtime tool expansion now reads schemas from current toolset Apps; decodes toolset schema storage.
packages/service/core/workflow/dispatch/ai/agent/sub/tool/index.ts Agent tool execution resolves toolset config/tool list from authorized toolset Apps at runtime.
packages/service/core/app/utils.ts Tool node hydration uses toolId derived from toolConfig when pluginId is absent.
packages/service/core/app/tool/utils/client.ts Client tool preview omits nested schema fields and reads MCP/HTTP tool definitions from toolset Apps.
packages/service/core/app/mcp.ts getMCPChildren now type-gated to MCP toolset Apps and supports legacy child-app format.
packages/service/core/app/http.ts Adds getHTTPToolList with type gating + schema-safe parsing.
packages/global/test/core/workflow/migration/schema.test.ts Adds tests for storage boundary schema stripping and Agent selected-tools schema stripping.
packages/global/core/workflow/utils.ts Uses NodeToolConfigStorageTypeSchema to strip schema fields when building toolset node IO.
packages/global/core/workflow/type/node.ts Adds runtime vs storage toolset config schemas + StoreWorkflowNodeItemTypeSchema.
packages/global/core/workflow/migration/schema.ts Migration parsing now uses storage tool-config schema for Agent selected tools.
packages/global/core/workflow/migration/legacy/workflow.ts Safer legacy headerSecret parsing during migration.
packages/global/core/app/tool/mcpTool/utils.ts Removes runtime mcpToolSet snapshot injection from MCP runtime node builder.
Review details
  • Files reviewed: 29/29 changed files
  • Comments generated: 3
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment on lines +36 to +38
customHeaders: z.string().optional().meta({
description: 'HTTP 工具集公共请求头 JSON 字符串'
}),
Comment on lines +190 to +196
const storageNodes =
storageModules ??
(type === AppTypeEnum.mcpToolSet
? encodeMcpToolSetNodesForStorage(normalizedWorkflow.nodes)
: type === AppTypeEnum.httpToolSet
? encodeHttpToolSetNodesForStorage(normalizedWorkflow.nodes)
: StoreWorkflowNodeItemTypeSchema.array().parse(normalizedWorkflow.nodes));
Comment on lines +581 to +593
const getAuthorizedToolSet = async (toolSetId: string) => {
try {
return (
await authAppByTmbId({
tmbId,
appId: toolSetId,
per: ReadPermissionVal
})
).app;
} catch {
return undefined;
}
};
@github-actions

github-actions Bot commented Sep 4, 2026

Copy link
Copy Markdown

Admin Preview Image Ready!

ghcr.io/labring/fastgpt-pr:admin_04a6bb53ed4df636e8ed2e064f52055d92a170c4

🕒 Time: 2026-09-07 13:09:28 (UTC+8)

@github-actions

github-actions Bot commented Sep 4, 2026

Copy link
Copy Markdown

Coverage Report

Status Category Percentage Covered / Total
🔵 Lines 33.89% 3407 / 10053
🔵 Statements 33.81% 3600 / 10647
🔵 Functions 28.6% 685 / 2395
🔵 Branches 28.57% 1713 / 5995
File CoverageNo changed files found.
Generated in workflow #3054 for commit 04a6bb5 by the Vitest Coverage Report Action

@github-actions

github-actions Bot commented Sep 4, 2026

Copy link
Copy Markdown

Build and publish successful - Preview fastgpt Image:

ghcr.io/labring/fastgpt-pr:fastgpt_04a6bb53ed4df636e8ed2e064f52055d92a170c4

🕒 Time: 2026-09-07 13:10:23 (UTC+8)

@c121914yu
c121914yu force-pushed the codex/pr-7634-runtime-schema branch from bc5641e to 72a3aa5 Compare September 7, 2026 02:23
@c121914yu
c121914yu force-pushed the codex/pr-7634-runtime-schema branch from 72a3aa5 to e7a15e8 Compare September 7, 2026 02:43
@c121914yu
c121914yu merged commit cfb8568 into labring:main Sep 7, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

mcp工具以$开头的字段无法存储

3 participants