diff --git a/.agents/design/core/app/app-resource-permission.md b/.agents/design/core/app/app-resource-permission.md new file mode 100644 index 000000000000..88fb7b464ece --- /dev/null +++ b/.agents/design/core/app/app-resource-permission.md @@ -0,0 +1,274 @@ +# 应用工作流与资源快照 + +目标:Version 成为工作流图和资源授权的唯一事实;App 只保留资源和元数据。不再用 `apps.modules` 当草稿/运行配置。`resources` 只写在 Version 上:缺字段时现场走与迁移相同的提取,`[]` 就是空快照。 + +相较上一版设计的关键变化: + +- 资源快照只落 `app_versions.resources`,App 主表不存这份缓存; +- 取消「正式 Version 缺 `resources` 抛 `App resources are not migrated`」; +- 工作流图迁到 Version,App 只用 `publishedVersionId` 指向正式版; +- 保存/发布鉴权改为相对最新 Version 的增量校验(第 7 节)。 + +资源模型(`AppResource`、工具归一化、`toolNames`、模型只统计不鉴权)与上一版相同,见文末附录。 + +--- + +## 1. 数据落在哪 + +### 1.1 `app_versions`(唯一事实) + +每条 Version 一次写入:`nodes / edges / chatConfig / resources`。 + +正式 Version 的 `resources` 必须和该条 nodes 同源。草稿(自动保存/普通保存)允许节点里出现尚未写入快照的新增引用,见第 7 节。 + +### 1.2 `apps`(资源行 + 指针,不再存图) + +留下:`name / avatar / intro / type / parentId / teamId / tmbId / permission 相关 / deleteTime / pluginData / scheduledTrigger*` 等。 + +增加(或复用现有指针): + +- `publishedVersionId`:当前最新正式 Version。反查、正式 Chat 选版都用它,避免扫全部历史正式版。 +- 编辑器工作副本不保存 App 指针,每次按该 App 的 `time` 最新 Version 读取(含 `isAutoSave`)。 + +去掉: + +- `modules / edges / chatConfig`(迁移并改读路径后 `$unset`) +- `resourceRefs`(本次重构前已存在,4163 `$unset`) + +文件夹没有工作流,不写 Version,也没有这些图字段。 + +`pluginData.nodeVersion` 与 `publishedVersionId` 的职责不得混用:正式运行只看 `publishedVersionId`(或等价的最新正式 Version 查询),不看 `nodeVersion`。 + +--- + +## 2. 读路径 + +| 场景 | 读谁 | +| --- | --- | +| 正式 Chat / OutLink / MCP 调 App / 定时任务 | `publishedVersionId` 对应 Version;没有指针则 `isPublish: true` + `time: -1` | +| 子 App / 工具钉死 `versionId` | `getAppVersionById`,只读那一条 | +| 打开编辑器、复制工作流 | 该 App `time` 最新 Version(含 autoSave) | +| Skill/资源反查(哪些 App 在用) | 查 **当前正式 Version**:`_id ∈ publishedVersionId` 且 `resources.$elemMatch`。禁止对所有 `isPublish: true` 做 elemMatch,否则旧正式版会把已删引用算进去 | + +Test/Debug:仍用请求体 nodes,服务端 `extract` + 按当前操作人鉴权,不读已发布快照,不接受客户端传 `resources`。 + +不要用 `apps.modules` 补运行快照。不要用最新正式 Version 的 `resources` 去跑另一条 Version 的 nodes。 + +编辑器详情 `GET /core/app/detail`、创建 `POST /core/app/create`、画布和工具编辑都直接用 `nodes`,与 Version 同名。不要再把 Version.nodes 映射成 App.modules。`apps.modules` 只作为 4163 `$unset` 的历史字段。 + +应用列表的 `hasInteractiveNode`(评测选应用过滤表单输入 / 用户选择)只扫当前 `publishedVersionId` 对应 Version 的 `nodes`,不读 `apps.modules`。 + +--- + +## 3. 写路径 + +| 操作 | Version | App | +| --- | --- | --- | +| 自动保存 | upsert `isAutoSave: true`,写入图 + 增量后的 `resources` | 更新 `updateTime`;**不改** `publishedVersionId` | +| 普通保存 | insert 非正式 Version,图 + 增量后的 `resources` | 更新 `updateTime`;**不改** `publishedVersionId` | +| 保存并发布 | insert `isPublish: true`,图 + 完整 `resources`(新增无权限则整次失败) | 更新 `publishedVersionId`、定时触发等 | +| 创建非文件夹 App | 首条 `isPublish: true` | 设好 `publishedVersionId` | +| 原地类型转换 | 必须写 Version(upsert autoSave 或 insert 非正式),直接复制最新 Version 的 `resources` 快照,不按转化操作者重新校验或过滤 | 不存图 | +| MCP/HTTP 更新 | `updateOne` **`publishedVersionId` 对应的那条** Version 的 `nodes` + `resources: []`(无指针回退 draft/最新) | 不存图;无产品版本历史 | + +保存/发布:`extractAppResources` → 相对上一版增量鉴权(第 7 节)→ 与 nodes 同事务写入。`model` 只记录,不鉴权。系统/商业工具不进快照。 + +原地类型转换是已有应用状态迁移,不属于普通保存:工作流转化写入的 autoSave Version 直接沿用转化前最新 Version 的 `resources` 快照,保持已确认的资源权限,不重新按当前操作者做宽松过滤。 + +普通保存/自动保存不得更新正式指针。 + +--- + +## 4. `resources` 怎么解析 + +对**当前这条 Version**: + +| 存储 | 含义 | 行为 | +| --- | --- | --- | +| 字段不存在 / `null` / 非数组 | 未迁 | 与 4163 `buildResources` 相同:对该条 `nodes + chatConfig` 做 `extractAppResources`;若仍有 `resourceRefs.skillIds` 则 merge | +| `[]` | 已迁或已保存,明确无引用 | 空快照,不再从 nodes 提取 | +| 合法非空数组 | 已有快照 | `safeParse` 后直接用 | +| 数组结构非法(`safeParse` 失败) | 视同缺字段 | 回退到该条 `nodes + chatConfig` 的 `extractAppResources`,避免单条脏数据炸链路;代价是可能重新纳入快照原本排除的引用 | + +不要抛 `App resources are not migrated`。 +不要把缺字段当成 `[]`。 + +允许读时懒回写 `$set.resources`,避免每次 extract。4163 仍可批量回填并 `$unset resourceRefs`。 + +空数组与缺字段必须区分:不引任何资源的应用保存后就是 `[]`。 + +--- + +## 5. 运行时授权 + +- 静态资源:必须命中当前选中 Version 的 `resources`,不按运行人 ACL。 +- 动态输入:`nodeHasDynamicInput` 标记后,按运行人 `auth*`。不能用「未命中快照」反推动态。 +- 父子不共享快照:父先过 `agent/tool`,子用子 Version 自己的 `resources`。 +- 系统 Skill 放行;MCP/HTTP 用父工具集 id + 可选 `toolNames`。 +- `authTmbId` 仍是终端用户数据过滤,和发布快照是两层。 +- 无 `resourceContext` 仅保留非 App 场景:Skill 调试、商业工具主动清空父快照。App 正式运行(含 Pro 评测 / Home Chat)必须带快照。不能裸 `findById`。 +- 入口批量加载实体;root Test/Debug 才允许跨团队,并向下传 `isRoot`。 +- 缺失/软删/跨团队**不在入口 fail-fast**:实体不在快照 map 时,只在用到该资源的节点按需抛错(`loadWorkflowDatasetResource` / `loadWorkflowAppResource` / `assertWorkflowResource`),其余节点照常执行。编辑器通过 workflow check 标出失效引用并定位到节点(`resource_missing`)。 + +--- + +## 6. 迁移 + +### 6.1 4163(资源) + +- 已有 Version:按该条 nodes 提取并写入 `resources`(逻辑与运行时缺字段相同)。 +- 有正式 Version:不要用 App.modules 覆盖该 Version 的 nodes。 +- 清 `resourceRefs`。 +- 给仍在用的 App 补 `publishedVersionId`(最新 `isPublish: true`)。 +- 正式指针回填 OCC:只在 `publishedVersionId` 仍为空或仍是本次扫描结果时写入,避免覆盖并发发布。 + +### 6.2 补 Version(仅零条记录) + +- `app_versions` 里该 `appId` **一条都没有**:才用 App 的 `modules/edges/chatConfig` 建一条 `isPublish: true`,并写出 `resources`、`publishedVersionId`。 +- **只要有任意 Version(含 MCP/HTTP 那一条、仅 autoSave)**:不把 App 图拷进 Version。 + +### 6.3 清 App 图 + +1. 补 Version + 4163 完成 +2. 读路径全部改到 Version +3. 写路径不再写 App 图 +4. `$unset modules/edges/chatConfig` 以及 `resourceRefs` + +**MCP/HTTP**:已有 Version,不新建、不覆盖;最多校验与旧 `apps.modules` 是否一致。 + +**类型转换**:清空前必须改为写 Version,否则「有 Version 就不迁 App 图」会丢掉只写在 App 上的转换结果。 + +--- + +## 7. 保存增量鉴权 + +动机:协作编辑时,上一版已经授权的资源不应要求当前保存人再具备读权限;否则改一句提示词也会被已撤权的知识库挡住。 + +比较基准统一为 **该 App `time` 最新 Version**(含自动保存记录)。缺 `resources` 时按第 4 节 extract 得到 baseline。 + +```text +extracted = extractAppResources(nodes, chatConfig) +baseline = 最新 Version 的 resources +added = extracted 相对 baseline 的新增(type + id,模型另含 modelType) +kept = extracted 中已在 baseline 出现的部分(不重新鉴权) +``` + +- 已在 baseline 中的资源:直接进入本版 `resources`。节点里删掉的引用不再保留。 +- 新增资源:按当前操作人做读权限校验。 +- `model` 不鉴权。系统/商业工具不进快照。 +- 创建 App(无上一版)视为全部新增,全量校验。 +- Test/Debug 仍使用请求体 nodes 生成运行时资源上下文;主应用先按当前操作人校验,工作流资源只校验相对 draft baseline 的新增部分。已在草稿快照中的资源不重复校验当前操作人。 + +### 7.1 自动保存 / 普通保存:不阻断 + +节点原样写入。`resources` 只收录: + +- `kept`(上一版已有,即使当前用户已无读权限) +- `added` 里当前用户**有**读权限的部分 + +无权限的新增引用留在 nodes 里,不写入 `resources`,保存成功。草稿允许 `nodes` 比 `resources` 多。 + +- 指向**已删除/不存在**实体的新增引用与无权限同口径:保存/自动保存不阻断,丢弃出 `resources`、留在 nodes;下次进入编辑器或轮询扫描时 workflow check 标出(`resource_missing`)并定位,直到用户移除节点,或发布时被 §7.2 阻断。 +- `kept`(已在 baseline 里的资源)不做存在性检查、不重验权限:引用从节点移除后 extract 自然不再产出,本版 `resources` 自动剔除;被删引用由编辑器轮询 / workflow check 标出并定位,提示用户移除节点。 + +### 7.2 保存并发布:阻断 + +相对保存开始时读取到的最新 Version 做增量。`added` 里任一无权限(含已删除/不存在,`unExist`),**整次发布失败**,指出缺权限的资源,不写正式 Version,不改 `publishedVersionId`。 + +全部新增都有权限时,写入的 `resources = kept + added`,与本次 nodes 同源。正式 Chat 只跑这类 Version。 + +含义:协作者可以把 owner 已经写进草稿快照的资源发布出去(相对 draft 不算新增);自己新加、自己没权限的资源过不了发布。 + +### 7.3 `toolNames` 不是权限层 + +本分支之前,MCP/HTTP 只对**父工具集 App** 做读权限,没有按子工具名鉴权。本方案不新增这一层。 + +增量 `added` 只按 `type + id`。父工具集已在 baseline 时,再勾子工具或改成整包都不触发鉴权。`toolNames` 随本次 extract 写入,只表示这版图选了哪些子工具(运行时按快照过滤),不是独立 ACL。 + +新拖进一个尚未在 baseline 里的工具集:算 `tool` id 新增,保存/发布按 7.1 / 7.2 对该父 App 做读权限校验。 + +--- + +## 8. 反查 + +Skill 列表 appCount、Skill 详情「被哪些 App 引用」: + +```text +apps: { _id, publishedVersionId, teamId, deleteTime } +join / $in publishedVersionId +app_versions.resources $elemMatch { type, id } +``` + +不要:`app_versions.find({ isPublish: true, resources: $elemMatch })` 再 distinct appId。 + +索引: + +- App:`{ teamId: 1, deleteTime: 1, publishedVersionId: 1 }` +- Version:现有 `{ appId: 1, time: -1 }`;另有 `{ appId, resources.type, resources.id }` 供 Version 自身资源查询 +- 历史 `{ teamId, deleteTime, resourceRefs.skillIds }` 声明 `deprecated: true`,由索引管理器清理 + +--- + +## 9. 落地顺序 + +1. **读容错**:`getAppLatestVersion` / `getAppVersionById` 对缺 `resources` 改为 extract,去掉该错误码;`[]` 保持空。 +2. **指针**:发布/创建写入 `publishedVersionId`;4163 回填。 +3. **反查改查正式 Version**。 +4. **零 Version 补建正式 Version**。 +5. 编辑器/复制/Chat 回退改读 Version;类型转换/MCP 更新只写 Version。 +6. `$unset` App 上的图字段和 `resourceRefs`。 + +第 1 步可先于清 App 上线,聊天不必等 4163 跑完。4–6 是结构重构,要按序,不能先清空再改读。 + +增量鉴权(第 7 节)与保存接口同一批改。编辑器读取和保存基准均按最新 Version 计算,不再维护草稿指针;4163 只回填正式 Version 指针。 + +--- + +## 10. 明确不做的事 + +- 用最新正式 Version 的 `resources` 去跑另一条 Version 的 nodes +- 用 App 工作副本给正式 Chat 授权 +- 普通保存/自动保存更新「正式」指针 +- 缺 `resources` 当 `[]` +- 把有 Version 的 App 的 `modules` 盖到正式 Version 上 +- 对所有历史 `isPublish: true` 做资源反查 +- 按 MCP/HTTP 子工具名做读权限(与改之前一致,权限只打到父工具集 App) + +--- + +## 11. 已确认决策 + +1. 读时懒回写 `resources` 本次不做。4163 批量回填即可;运行时缺字段每次 extract。 +2. `pluginData.nodeVersion` 继续写,仅给工具/子 App 钉版本用。正式运行只看 `publishedVersionId`。 + +--- + +## 12. 实施 TODO + +- [x] `getAppLatestVersion` / `getAppVersionById` 缺字段 extract,删除 `App resources are not migrated`。 +- [x] App 增加并维护 `publishedVersionId`(创建/发布/4163)。 +- [x] 反查改为正式 Version `$elemMatch`。 +- [x] 保存增量鉴权(相对 baseline,旧资源不重验)。 +- [x] 无 `resourceContext` 时 dataset/toolset 走运行人鉴权,禁止裸 `findById`。 +- [x] 零 Version 补建正式 Version;有 Version 不覆盖。 +- [x] 类型转换/MCP 更新只写 Version。 +- [x] 编辑器与复制改读 draft/最新 Version。 +- [x] `$unset` App 图字段和 `resourceRefs`;废弃 `resourceRefs.skillIds` 索引已登记。 +- [x] 测试:缺字段 extract vs `[]`;反查只计当前正式版。其余:发布后撤权仍能跑正式版;协作者保存不丢旧资源;新增无权限按确认口径;嵌套钉版本只读那一条。 + +--- + +## 附录:资源模型(沿用) + +```ts +type AppResource = + | { type: 'tool'; id: string; data?: { toolNames?: string[] } } + | { type: 'model'; id: string; data: { modelType: 'llm' | 'rerank' | 'tts' } } + | { type: 'agent' | 'dataset' | 'skill'; id: string }; +``` + +- `tool`:个人工作流工具、MCP/HTTP 工具集;系统/商业工具不进快照。 +- `data.toolNames`:这版图选中的 MCP/HTTP 子工具,缺省表示整包。只用于运行时按快照过滤,**不是**读权限粒度;鉴权只针对父工具集 App。 +- `model` 只统计,不鉴权;动态模型输入不写入静态快照。 +- 提取器只解析、归一化、合并、去重、稳定排序,不访问数据库。 +- 动态资源 ID 保存阶段不确定则不虚构记录。 diff --git a/document/data/doc-last-modified.json b/document/data/doc-last-modified.json index f9fecd0a2b3b..2bc02a2a8712 100644 --- a/document/data/doc-last-modified.json +++ b/document/data/doc-last-modified.json @@ -501,4 +501,4 @@ "content/self-host/upgrading/version-timeline.mdx": "2026-08-16T23:16:43+08:00", "content/toc.en.mdx": "2026-09-03T21:50:14+08:00", "content/toc.mdx": "2026-09-03T21:50:14+08:00" -} \ No newline at end of file +} diff --git a/packages/global/core/app/formEdit/type.ts b/packages/global/core/app/formEdit/type.ts index 1db7526cf7c2..0f03c534f574 100644 --- a/packages/global/core/app/formEdit/type.ts +++ b/packages/global/core/app/formEdit/type.ts @@ -13,7 +13,9 @@ export const SelectedAgentSkillItemTypeSchema = z.object({ name: z.string(), description: z.string().default(''), avatar: z.string().optional(), - isDeleted: z.boolean().default(false) + isDeleted: z.boolean().default(false), + /** 实体存在但不在当前版本资源快照内(保存时被无权限丢弃),运行时 assertWorkflowResource 会拒绝。 */ + permissionDenied: z.boolean().optional() }); export type SelectedAgentSkillItemType = z.infer; export const StoredSelectedAgentSkillItemTypeSchema = SelectedAgentSkillItemTypeSchema.pick({ diff --git a/packages/global/core/app/type.ts b/packages/global/core/app/type.ts index b8210555fbaf..2e0db83890f3 100644 --- a/packages/global/core/app/type.ts +++ b/packages/global/core/app/type.ts @@ -1,9 +1,7 @@ -import { StoreNodeItemTypeSchema } from '../workflow/type/node'; import { AppTypeEnum } from './constants'; import { NodeInputKeyEnum } from '../workflow/constants'; import { DatasetSearchModeEnum } from '../dataset/constants'; import type { ReasoningEffort } from '../ai/llm/type'; -import { StoreEdgeItemTypeSchema } from '../workflow/type/edge'; import type { AppPermission } from '../../support/permission/app/controller'; import { ParentIdSchema, type ParentIdType } from '../../common/parentFolder/type'; import type { WorkflowTemplateBasicType } from '../workflow/type'; @@ -13,6 +11,7 @@ import { ObjectIdSchema } from '../../common/type/mongo'; import { AppFileSelectConfigTypeSchema } from './type/config.schema'; import { BoolSchema, NumSchema } from '../../common/zod'; import { VariableItemTypeSchema } from './variable/type'; +import type { AppVersionSchemaType } from './version/type'; // tts export const AppTTSConfigTypeSchema = z.object({ @@ -150,12 +149,40 @@ export const AppChatConfigTypeSchema = z.object({ }); export type AppChatConfigType = z.infer; -export const AppResourceRefsSchema = z.object({ - skillIds: z.array(z.string()).default([]).meta({ - description: '应用发布版本引用的技能 ID 列表' +const AppResourceToolDataSchema = z.object({ + toolNames: z.array(z.string()).optional().meta({ + description: 'MCP/HTTP 工具集允许执行的子工具名;为空表示整个工具集' }) }); -export type AppResourceRefsType = z.infer; + +const AppResourceModelDataSchema = z.object({ + modelType: z.enum(['llm', 'rerank', 'tts']).meta({ + description: '模型资源类型' + }) +}); + +export const AppResourceSchema = z.discriminatedUnion('type', [ + z.object({ + type: z.literal('tool'), + id: z.string(), + data: AppResourceToolDataSchema.optional() + }), + z.object({ + type: z.literal('model'), + id: z.string(), + data: AppResourceModelDataSchema + }), + z.object({ + type: z.enum(['agent', 'dataset', 'skill']), + id: z.string() + }) +]); +export type AppResource = z.infer; +export const AppResourceTypeSchema = z.enum(['tool', 'model', 'agent', 'dataset', 'skill']); +export type AppResourceType = z.infer; + +export const AppResourcesSchema = z.array(AppResourceSchema); +export type AppResourcesType = z.infer; // Mongo Collection export const AppStorageSchemaTypeSchema = z.object({ @@ -174,8 +201,6 @@ export const AppStorageSchemaTypeSchema = z.object({ updateTime: z.coerce.date(), createTime: z.coerce.date().optional(), - modules: z.array(StoreNodeItemTypeSchema), - edges: z.array(StoreEdgeItemTypeSchema), pluginData: z .object({ nodeVersion: z.string().optional().meta({ @@ -196,11 +221,11 @@ export const AppStorageSchemaTypeSchema = z.object({ description: '应用扩展配置,主要用于工具集和旧版插件应用' }), - // App system config - chatConfig: AppChatConfigTypeSchema, scheduledTriggerConfig: AppScheduledTriggerConfigTypeSchema.optional(), scheduledTriggerNextTime: z.coerce.date().optional(), - resourceRefs: AppResourceRefsSchema.optional(), + publishedVersionId: ObjectIdSchema.optional().meta({ + description: '当前最新正式发布 Version ID' + }), inheritPermission: BoolSchema.optional(), // if access the app by favourite or quick @@ -248,7 +273,16 @@ export type AppListItemType = { hasInteractiveNode?: boolean; }; +/** 鉴权得到的 App 行:Mongo 元数据 + 权限。工作流图在 Version 上。 */ +export type AppWithPermissionType = AppSchemaType & { + permission: AppPermission; +}; + +/** 应用详情:元数据 + 当前草稿 Version 的 nodes/edges/chatConfig。 */ export type AppDetailType = AppSchemaType & { + nodes: AppVersionSchemaType['nodes']; + edges: AppVersionSchemaType['edges']; + chatConfig: AppVersionSchemaType['chatConfig']; permission: AppPermission; }; diff --git a/packages/global/core/app/version/type.ts b/packages/global/core/app/version/type.ts index 2c000994bff0..78f59bef931f 100644 --- a/packages/global/core/app/version/type.ts +++ b/packages/global/core/app/version/type.ts @@ -1,5 +1,6 @@ -import { AppSchemaTypeSchema } from '../type'; -import { AppResourceRefsSchema } from '../type'; +import { AppResourcesSchema, AppChatConfigTypeSchema } from '../type'; +import { StoreNodeItemTypeSchema } from '../../workflow/type/node'; +import { StoreEdgeItemTypeSchema } from '../../workflow/type/edge'; import { SourceMemberSchema } from '../../../support/user/type'; import z from 'zod'; import { ObjectIdSchema } from '../../../common/type/mongo'; @@ -9,13 +10,13 @@ export const AppVersionSchema = z.object({ tmbId: ObjectIdSchema, appId: ObjectIdSchema, time: z.coerce.date(), - nodes: AppSchemaTypeSchema.shape.modules, - edges: AppSchemaTypeSchema.shape.edges, - chatConfig: AppSchemaTypeSchema.shape.chatConfig, + nodes: z.array(StoreNodeItemTypeSchema), + edges: z.array(StoreEdgeItemTypeSchema), + chatConfig: AppChatConfigTypeSchema, isPublish: z.boolean().optional(), isAutoSave: z.boolean().optional(), versionName: z.string(), - resourceRefs: AppResourceRefsSchema.optional() + resources: AppResourcesSchema.optional() }); export type AppVersionSchemaType = z.infer; @@ -37,9 +38,9 @@ export const PublishAppQuerySchema = z.object({ export type PublishAppQueryType = z.infer; export const PublishAppBodySchema = z.object({ - nodes: AppSchemaTypeSchema.shape.modules.optional(), - edges: AppSchemaTypeSchema.shape.edges.optional(), - chatConfig: AppSchemaTypeSchema.shape.chatConfig.optional(), + nodes: AppVersionSchema.shape.nodes.optional(), + edges: AppVersionSchema.shape.edges.optional(), + chatConfig: AppVersionSchema.shape.chatConfig.optional(), isPublish: z.boolean().optional(), versionName: z.string().optional(), autoSave: z.boolean().optional() diff --git a/packages/global/core/workflow/type/io.ts b/packages/global/core/workflow/type/io.ts index 91a085afe726..da0394604e5c 100644 --- a/packages/global/core/workflow/type/io.ts +++ b/packages/global/core/workflow/type/io.ts @@ -21,7 +21,9 @@ export const SelectedDatasetSchema = z.object({ description: '知识库使用的向量模型' }) }), - isDeleted: BoolSchema.optional() + isDeleted: BoolSchema.optional(), + /** 实体存在但不在当前版本资源快照内(保存时被无权限丢弃),运行时 assertWorkflowResource 会拒绝。 */ + permissionDenied: BoolSchema.optional() }); export type SelectedDatasetType = z.infer; diff --git a/packages/global/core/workflow/type/node.ts b/packages/global/core/workflow/type/node.ts index 64d62a50f666..a568fbf027a3 100644 --- a/packages/global/core/workflow/type/node.ts +++ b/packages/global/core/workflow/type/node.ts @@ -164,6 +164,8 @@ export const ToolDataSchema = z.object({ error: z.string().optional().meta({ description: '工具配置错误说明' }), + /** 工具实体存在但不在当前版本资源快照内(保存时被无权限丢弃),运行时 assertWorkflowResource 会拒绝。 */ + permissionDenied: BoolSchema.optional(), status: PluginStatusSchema.optional().meta({ description: '工具当前状态' }) diff --git a/packages/global/core/workflow/utils.ts b/packages/global/core/workflow/utils.ts index cbabfe43ec9b..c558eb30d0e0 100644 --- a/packages/global/core/workflow/utils.ts +++ b/packages/global/core/workflow/utils.ts @@ -21,7 +21,7 @@ import { } from './type/io'; import type { NodeToolConfigType, StoreNodeItemType } from './type/node'; import { ToolSetToolSummarySchema } from '../app/tool/toolSet/type'; -import type { AppChatConfigType, AppSchemaType, AppWelcomeConfigType } from '../app/type'; +import type { AppChatConfigType, AppWelcomeConfigType } from '../app/type'; import type { VariableItemType } from '../app/variable/type'; import { normalizeAndParseVariableList } from '../app/variable/utils'; import { type EditorVariablePickerType } from '../../../web/components/common/Textarea/PromptEditor/type'; @@ -634,7 +634,7 @@ export const formatModels = ({ modelReferencePolicy }: { nodes: StoreNodeItemType[] | undefined; - chatConfig?: AppSchemaType['chatConfig']; + chatConfig?: AppChatConfigType; models?: Array<{ modelId: string; model: string; type: ModelTypeEnum }>; defaultModelIds?: Partial>; modelReferencePolicy: 'preserve' | 'fallback' | 'validate' | 'import'; @@ -890,7 +890,7 @@ export const addModelNamesToWorkflow = ({ models = [] }: { nodes?: StoreNodeItemType[]; - chatConfig?: AppSchemaType['chatConfig']; + chatConfig?: AppChatConfigType; models?: Array<{ modelId: string; model: string; type: ModelTypeEnum }>; }) => { const findModelName = ({ modelId, type }: { modelId: unknown; type: ModelTypeEnum }) => { diff --git a/packages/global/openapi/core/app/common/api.ts b/packages/global/openapi/core/app/common/api.ts index b83b263bc08b..1b7236ef8dc9 100644 --- a/packages/global/openapi/core/app/common/api.ts +++ b/packages/global/openapi/core/app/common/api.ts @@ -4,7 +4,6 @@ import { AppListSortEnum, AppTypeEnum } from '../../../../core/app/constants'; import { AppChatConfigTypeSchema, AppTTSConfigTypeSchema, - AppResourceRefsSchema, AppScheduledTriggerConfigTypeSchema, AppSchemaTypeSchema, type AppDetailType, @@ -178,12 +177,14 @@ const migrateCreateAppBodyWorkflow = (value: unknown) => { chatConfig: body.chatConfig }); - return { + const migratedBody: Record = { ...body, - modules: workflow.nodes, + nodes: workflow.nodes, edges: workflow.edges, chatConfig: workflow.chatConfig }; + delete migratedBody.modules; + return migratedBody; } catch { return value; } @@ -218,10 +219,12 @@ export const CreateAppBodySchema = z example: AppTypeEnum.workflow, description: '应用类型' }), - modules: z.array(CreateAppNodeSchema).optional().meta({ + nodes: z.array(CreateAppNodeSchema).optional().meta({ example: [], description: '应用节点配置' }), + // 旧 modules 只能在 preprocess 成功后被删除;残留时必须拒绝,避免静默创建空应用。 + modules: z.never().optional(), edges: CreateAppEdgesSchema.optional().meta({ example: [], description: '应用连线' @@ -241,7 +244,7 @@ export const CreateAppBodySchema = z example: { name: '新应用', type: AppTypeEnum.simple, - modules: [], + nodes: [], edges: [], parentId: '68ad85a7463006c963799a05' } @@ -371,11 +374,8 @@ export const GetAppDetailResponseSchema = AppSchemaTypeSchema.extend({ description: '创建应用时使用的模板 ID' }), updateTime: z.coerce.date().meta({ description: '最后更新时间' }), - modules: z - .array(OpenAPIStoreNodeItemTypeSchema) - .default([]) - .meta({ description: '应用节点配置' }), - edges: AppSchemaTypeSchema.shape.edges.default([]).meta({ + nodes: z.array(OpenAPIStoreNodeItemTypeSchema).default([]).meta({ description: '应用节点配置' }), + edges: z.array(StoreEdgeItemTypeSchema).default([]).meta({ description: '应用连线' }), pluginData: AppSchemaTypeSchema.shape.pluginData, @@ -388,8 +388,8 @@ export const GetAppDetailResponseSchema = AppSchemaTypeSchema.extend({ scheduledTriggerNextTime: z.coerce.date().optional().meta({ description: '下一次定时触发时间' }), - resourceRefs: AppResourceRefsSchema.optional().meta({ - description: '应用发布后引用的外部资源集合' + publishedVersionId: ObjectIdSchema.optional().meta({ + description: '当前最新正式发布 Version ID' }), inheritPermission: BoolSchema.optional().meta({ description: '是否继承父级文件夹权限' diff --git a/packages/global/openapi/core/app/httpTools/api.ts b/packages/global/openapi/core/app/httpTools/api.ts index 64a7bcbc4f52..f3900023c759 100644 --- a/packages/global/openapi/core/app/httpTools/api.ts +++ b/packages/global/openapi/core/app/httpTools/api.ts @@ -11,7 +11,7 @@ import { HttpToolTypeEnum } from '../../../../core/app/tool/httpTool/constants'; * ============================================================================ */ export const CreateHttpToolsBodySchema = CreateAppBodySchema.omit({ type: true, - modules: true, + nodes: true, edges: true, chatConfig: true }) diff --git a/packages/global/openapi/core/app/mcpTools/api.ts b/packages/global/openapi/core/app/mcpTools/api.ts index 3dd238ee5730..b287eb13e0ce 100644 --- a/packages/global/openapi/core/app/mcpTools/api.ts +++ b/packages/global/openapi/core/app/mcpTools/api.ts @@ -7,7 +7,7 @@ import { McpToolConfigSchema } from '../../../../core/app/tool/mcpTool/type'; // Create mcp tool export const CreateMcpToolsBodySchema = CreateAppBodySchema.omit({ type: true, - modules: true, + nodes: true, edges: true, chatConfig: true }) diff --git a/packages/global/openapi/core/app/version/api.ts b/packages/global/openapi/core/app/version/api.ts index a8861597bd49..b1f509a866fb 100644 --- a/packages/global/openapi/core/app/version/api.ts +++ b/packages/global/openapi/core/app/version/api.ts @@ -3,7 +3,8 @@ import { VersionListItemSchema } from '../../../../core/app/version/type'; import { ObjectIdSchema } from '../../../../common/type/mongo'; import { PaginationSchema } from '../../../api'; import { OpenAPIStoreNodeItemTypeSchema } from '../../workflow/node'; -import { AppResourceRefsSchema, AppSchemaTypeSchema } from '../../../../core/app/type'; +import { AppResourcesSchema } from '../../../../core/app/type'; +import { StoreEdgeItemTypeSchema } from '../../../../core/workflow/type/edge'; import { AppChatConfigInputSchema, OpenAPIAppChatConfigSchema } from '../common/api'; import { BoolSchema, NumSchema } from '../../../../common/zod'; @@ -23,7 +24,7 @@ const PublishAppNodesSchema = z.array(OpenAPIStoreNodeItemTypeSchema).meta({ description: '本次保存的应用节点配置' }); -const AppVersionEdgesSchema = AppSchemaTypeSchema.shape.edges.default([]).meta({ +const AppVersionEdgesSchema = z.array(StoreEdgeItemTypeSchema).default([]).meta({ description: '版本内保存的应用连线配置' }); @@ -34,8 +35,8 @@ const AppVersionChatConfigInputSchema = AppChatConfigInputSchema.default({}).met description: '本次写入的应用对话配置' }); -const AppVersionResourceRefsSchema = AppResourceRefsSchema.optional().meta({ - description: '该版本引用的外部资源集合' +const AppVersionResourcesSchema = AppResourcesSchema.meta({ + description: '该版本引用的资源集合' }); const OpenAPIVersionListItemSchema = VersionListItemSchema.extend({ @@ -182,7 +183,7 @@ export const GetAppVersionDetailResponseSchema = z.object({ example: '正式发布版', description: '版本名称' }), - resourceRefs: AppVersionResourceRefsSchema + resources: AppVersionResourcesSchema }); export type GetAppVersionDetailResponseType = z.infer; @@ -218,7 +219,8 @@ export const GetLatestAppVersionResponseSchema = z.object({ description: '版本内保存的应用节点配置' }), edges: AppVersionEdgesSchema, - chatConfig: AppVersionChatConfigSchema + chatConfig: AppVersionChatConfigSchema, + resources: AppVersionResourcesSchema }); export type GetLatestAppVersionResponseType = z.infer; diff --git a/packages/global/test/openapi/core/app/common/api.test.ts b/packages/global/test/openapi/core/app/common/api.test.ts index 68e5eb58c646..8245d4691778 100644 --- a/packages/global/test/openapi/core/app/common/api.test.ts +++ b/packages/global/test/openapi/core/app/common/api.test.ts @@ -35,7 +35,7 @@ describe('CreateAppBodySchema', () => { CreateAppBodySchema.safeParse({ name: 'canonical app', type: 'simple', - modules: [currentNode], + nodes: [currentNode], edges: [], chatConfig: {} }).success @@ -82,8 +82,8 @@ describe('CreateAppBodySchema', () => { chatConfig: { _id: 'legacy-chat-config' } }); - expect(result.modules).toHaveLength(1); - expect(result.modules?.[0].inputs[0]).not.toHaveProperty('llmModelType'); + expect(result.nodes).toHaveLength(1); + expect(result.nodes?.[0].inputs[0]).not.toHaveProperty('llmModelType'); expect(result.chatConfig).not.toHaveProperty('_id'); }); @@ -98,6 +98,18 @@ describe('CreateAppBodySchema', () => { }); expect(result).not.toHaveProperty('unsupported'); + expect(result.nodes).toHaveLength(1); + }); + + it('rejects legacy workflow when migration fails instead of creating an empty app', () => { + const result = CreateAppRequestBodySchema.safeParse({ + name: 'invalid legacy app', + type: 'simple', + modules: [{ flowType: 'legacy', moduleId: 'legacy-module' }], + edges: [] + }); + + expect(result.success).toBe(false); }); }); diff --git a/packages/service/core/ai/sandbox/application/runtime/skill/core.ts b/packages/service/core/ai/sandbox/application/runtime/skill/core.ts index 9c40075ea4d3..bb1235c5c7e4 100644 --- a/packages/service/core/ai/sandbox/application/runtime/skill/core.ts +++ b/packages/service/core/ai/sandbox/application/runtime/skill/core.ts @@ -14,7 +14,10 @@ import type { DeployedSkillInfo, DeployedSkillVersion } from './types'; import { getAgentSandboxSkillMaxBytes } from '../../../config'; import { joinSandboxPath } from '../../../utils'; import { authSkillByTmbId } from '../../../../../../support/permission/skill/auth'; +import { AgentSkillSourceEnum } from '@fastgpt/global/core/ai/skill/constants'; import { ReadPermissionVal } from '@fastgpt/global/support/permission/constant'; +import { getWorkflowResourceContext } from '../../../../../workflow/utils/context'; +import { assertWorkflowResource } from '../../../../../workflow/utils/resource'; import { SkillErrEnum } from '@fastgpt/global/common/error/code/skill'; import { Readable } from 'node:stream'; @@ -137,13 +140,15 @@ export const injectAgentSkillFilesToSandbox = async ({ skillIds, teamId, tmbId, - workDirectory + workDirectory, + dynamic = false }: { sandbox: ISandbox; skillIds: string[]; teamId: string; tmbId: string; workDirectory: string; + dynamic?: boolean; }): Promise => { const skillsRootPath = getRuntimeSkillsRootPath(workDirectory); await sandbox.createDirectories([skillsRootPath]); @@ -178,11 +183,26 @@ export const injectAgentSkillFilesToSandbox = async ({ return []; } - const teamSkills = await MongoAgentSkills.find({ - _id: { $in: skillIds }, - teamId, - deleteTime: null - }); + const resourceContext = getWorkflowResourceContext(); + if (resourceContext && !dynamic) { + skillIds.forEach((skillId) => + assertWorkflowResource({ + context: resourceContext, + type: 'skill', + id: skillId + }) + ); + } + const teamSkills = + resourceContext && !dynamic + ? skillIds + .map((skillId) => resourceContext.skillMap.get(skillId)) + .filter((skill): skill is NonNullable => !!skill) + : await MongoAgentSkills.find({ + _id: { $in: skillIds }, + deleteTime: null, + $or: [{ teamId }, { source: AgentSkillSourceEnum.system }] + }); if (teamSkills.length === 0) { logger.warn('[Agent Skills] No valid skills found from input skillIds', { skillIds }); await cleanupStaleDirs(new Set()); @@ -193,11 +213,19 @@ export const injectAgentSkillFilesToSandbox = async ({ await Promise.all( teamSkills.map(async (skill) => { try { - await authSkillByTmbId({ - tmbId, - skillId: String(skill._id), - per: ReadPermissionVal - }); + if (resourceContext && !dynamic) { + assertWorkflowResource({ + context: resourceContext, + type: 'skill', + id: String(skill._id) + }); + } else { + await authSkillByTmbId({ + tmbId, + skillId: String(skill._id), + per: ReadPermissionVal + }); + } return skill; } catch (error) { if (error !== SkillErrEnum.unAuthSkill && error !== SkillErrEnum.unExist) { diff --git a/packages/service/core/ai/skill/debugChat/handler.ts b/packages/service/core/ai/skill/debugChat/handler.ts index 653f1ce98614..5cd11a5e0c92 100644 --- a/packages/service/core/ai/skill/debugChat/handler.ts +++ b/packages/service/core/ai/skill/debugChat/handler.ts @@ -249,6 +249,7 @@ export async function handleSkillDebugChat( persistToDb: true, retainInMemory: true }, + // Skill 调试不是 App 运行,不带 Version 快照;静态资源按运行人 tmbId 鉴权。 agentSandboxPrepareActions: options.agentSandboxPrepareActions }); diff --git a/packages/service/core/ai/skill/manage/list.ts b/packages/service/core/ai/skill/manage/list.ts index 2fa7cf4ee16f..64bae7bd1e8a 100644 --- a/packages/service/core/ai/skill/manage/list.ts +++ b/packages/service/core/ai/skill/manage/list.ts @@ -1,9 +1,3 @@ -import { Types } from '../../../../common/mongo'; -import { MongoApp } from '../../../app/schema'; -import { AppResourceRefsSkillIdsPath, buildAppSkillRefMongoQuery } from '../../../app/resourceRefs'; -import { MongoAgentSkills } from '../model/schema'; -import { SkillPermission } from '@fastgpt/global/support/permission/skill/controller'; -import { PerResourceTypeEnum } from '@fastgpt/global/support/permission/constant'; import { getResourcePermissionsByTeam } from '../../../../support/permission/resourcePermissionService'; import { parseParentIdInMongo } from '@fastgpt/global/common/parentFolder/utils'; import { replaceRegChars } from '@fastgpt/global/common/string/tools'; @@ -15,6 +9,10 @@ import type { AgentSkillCreationStatusEnum } from '@fastgpt/global/core/ai/skill import { AgentSkillSourceEnum, AgentSkillTypeEnum } from '@fastgpt/global/core/ai/skill/constants'; import type { ListSkillsQuery } from '@fastgpt/global/core/ai/skill/api'; import { AppListSortEnum, appListSortMongoMap } from '@fastgpt/global/core/app/constants'; +import { MongoAgentSkills } from '../model/schema'; +import { SkillPermission } from '@fastgpt/global/support/permission/skill/controller'; +import { PerResourceTypeEnum } from '@fastgpt/global/support/permission/constant'; +import { findTeamAppsByPublishedResource } from '../../../app/resourceLookup'; type TeamPermission = { isOwner: boolean; @@ -163,7 +161,19 @@ export const listReadableAgentSkills = async ({ $or: [{ teamId }, { source: AgentSkillSourceEnum.system }] }; const idList = { _id: { $in: myRoleResourceIds } }; - const readPermissionQuery = teamPer.isOwner || source === 'store' ? {} : idList; + const readPermissionQuery = + teamPer.isOwner || source === 'store' + ? {} + : { + $or: [ + idList, + { tmbId }, + { + inheritPermission: true, + parentId: { $in: myRoleResourceIds } + } + ] + }; return mergeMongoAndQuery(baseQuery, scopeQuery, readPermissionQuery, { _id: { $in: selectedSkillIds }, @@ -171,9 +181,15 @@ export const listReadableAgentSkills = async ({ }); } - // 普通列表查询同时按当前目录和资源自身 ACL 过滤。 + // 普通列表查询需要叠加当前目录过滤;skillIds 查询已在上方按读权限过滤,但不受目录限制。 const idList = { _id: { $in: myRoleResourceIds } }; - const skillPerQuery = teamPer.isOwner || source === 'store' ? {} : idList; + const skillPerQuery = teamPer.isOwner + ? {} + : parentId + ? { + $or: [idList, parseParentIdInMongo(parentId)] + } + : { $or: [idList, { parentId: null }] }; const teamIdQuery = source === 'store' ? {} : { teamId }; if (searchKey) { @@ -207,6 +223,12 @@ export const listReadableAgentSkills = async ({ return roleCountByResourceId.get(skillId) ?? 0; }; + if (skill.inheritPermission && skill.parentId && skill.type !== AgentSkillTypeEnum.folder) { + return { + Per: getPer(String(skill.parentId)).addRole(getPer(String(skill._id)).role), + privateSkill: getClbCount(String(skill.parentId)) <= 1 + }; + } return { Per: getPer(String(skill._id)), privateSkill: getClbCount(String(skill._id)) <= 1 @@ -251,25 +273,13 @@ export const listReadableAgentSkills = async ({ const appCountMap = new Map(); if (nonFolderSkills.length > 0) { const skillIdStrings = nonFolderSkills.map((skill) => String(skill._id)); - const counts = await MongoApp.aggregate<{ _id: string; count: number }>([ - { - $match: { - teamId: new Types.ObjectId(String(teamId)), - deleteTime: null, - ...buildAppSkillRefMongoQuery(skillIdStrings) - } - }, - { $unwind: `$${AppResourceRefsSkillIdsPath}` }, - { $match: buildAppSkillRefMongoQuery(skillIdStrings) }, - { - $group: { - _id: `$${AppResourceRefsSkillIdsPath}`, - count: { $sum: 1 } - } - } - ]); - counts.forEach((item) => { - appCountMap.set(String(item._id), item.count); + const { counts } = await findTeamAppsByPublishedResource({ + teamId, + type: 'skill', + ids: skillIdStrings + }); + counts.forEach((count, skillId) => { + appCountMap.set(skillId, count); }); } diff --git a/packages/service/core/app/controller.ts b/packages/service/core/app/controller.ts index ed0f4514a732..3a51d8bcd480 100644 --- a/packages/service/core/app/controller.ts +++ b/packages/service/core/app/controller.ts @@ -1,6 +1,6 @@ import { type AppSchemaType } from '@fastgpt/global/core/app/type'; -import { AppTypeEnum } from '@fastgpt/global/core/app/constants'; import { NodeInputKeyEnum } from '@fastgpt/global/core/workflow/constants'; +import { AppTypeEnum } from '@fastgpt/global/core/app/constants'; import { FlowNodeTypeEnum } from '@fastgpt/global/core/workflow/node/constant'; import { SystemToolSecretInputTypeEnum } from '@fastgpt/global/core/app/tool/systemTool/constants'; import { MongoApp } from './schema'; @@ -16,10 +16,7 @@ import { MongoChatInputGuide } from '../chat/inputGuide/schema'; import { MongoChatFavouriteApp } from '../chat/favouriteApp/schema'; import { MongoChatSetting } from '../chat/setting/schema'; import { resourcePermissionRepo } from '../../support/permission/repository/resourcePermissionRepo'; -import { - PerResourceTypeEnum, - ReadPermissionVal -} from '@fastgpt/global/support/permission/constant'; +import { PerResourceTypeEnum } from '@fastgpt/global/support/permission/constant'; import { removeImageByPath } from '../../common/file/image/controller'; import { MongoAppLogKeys } from './logs/logkeysSchema'; import { MongoAppChatLog } from './logs/chatLogsSchema'; @@ -36,7 +33,6 @@ import { } from '@fastgpt/global/core/app/formEdit/type'; import z from 'zod'; import { nodeInputIsReference } from '@fastgpt/global/core/workflow/utils'; -import { authSkillByTmbId } from '../../support/permission/skill/auth'; import { ChatSourceTypeEnum } from '@fastgpt/global/core/chat/constants'; import { deleteChatResourcesBySource } from '../chat/delete'; @@ -163,45 +159,6 @@ export const beforeUpdateAppFormat = async ({ ); }; -/** - * 发布应用前校验静态绑定的 Agent Skill 对当前成员可读。 - * 引用输入在发布阶段没有确定值,运行时会按实际值再次过滤。 - */ -export const validatePublishAppAgentSkillReadPermissions = async ({ - nodes, - tmbId, - isRoot = false -}: { - nodes?: StoreNodeItemType[]; - tmbId: string; - isRoot?: boolean; -}) => { - if (!nodes) return; - - const skillIds = new Set(); - for (const node of nodes) { - for (const input of node.inputs) { - if (input.key !== NodeInputKeyEnum.skills || nodeInputIsReference(input)) continue; - - const skills = z.array(StoredSelectedAgentSkillItemTypeSchema).parse(input.value); - for (const skill of skills) { - skillIds.add(skill.skillId); - } - } - } - - await Promise.all( - Array.from(skillIds).map((skillId) => - authSkillByTmbId({ - tmbId, - skillId, - per: ReadPermissionVal, - isRoot - }) - ) - ); -}; - /* Get apps */ export async function findAppAndAllChildren({ teamId, diff --git a/packages/service/core/app/http.ts b/packages/service/core/app/http.ts index 89476c84efd4..c157d5505224 100644 --- a/packages/service/core/app/http.ts +++ b/packages/service/core/app/http.ts @@ -19,6 +19,7 @@ import { decodeHttpToolSetNodesFromStorage } from './jsonSchemaStorage'; import { buildOpenAPIHttpRequest } from './httpTool/request'; import { str2OpenApiSchema } from '@fastgpt/global/core/app/jsonschema'; import { completeOpenAPIRequestSchema } from '@fastgpt/global/core/app/tool/httpTool/utils'; +import { getAppLatestVersion, type AppPublishedWorkflow } from './version/controller'; const logger = getLogger(LogCategories.MODULE.APP.HTTP_TOOLS); @@ -208,11 +209,13 @@ export const runHTTPTool = async ({ }; /** Read the current HTTP tool list from a toolset app. */ -export const getHTTPToolList = async (app: AppSchemaType) => { +export const getHTTPToolList = async (app: AppSchemaType, workflow?: AppPublishedWorkflow) => { if (app.type !== AppTypeEnum.httpToolSet) return []; - const modules = decodeHttpToolSetNodesFromStorage(app.modules); - const toolSet = modules[0]?.toolConfig?.httpToolSet; + const nodes = decodeHttpToolSetNodesFromStorage( + (workflow ?? (await getAppLatestVersion(String(app._id), app))).nodes + ); + const toolSet = nodes[0]?.toolConfig?.httpToolSet; const toolList = HttpToolConfigTypeSchema.array().safeParse( toolSet && 'toolList' in toolSet ? toolSet.toolList : undefined ).data; diff --git a/packages/service/core/app/mcp.ts b/packages/service/core/app/mcp.ts index 017583cfeb60..ccd17621eb8d 100644 --- a/packages/service/core/app/mcp.ts +++ b/packages/service/core/app/mcp.ts @@ -6,22 +6,17 @@ import { } from '@modelcontextprotocol/sdk/client/streamableHttp.js'; import type { AppSchemaType } from '@fastgpt/global/core/app/type'; import { type McpToolConfigType } from '@fastgpt/global/core/app/tool/mcpTool/type'; -import { - SecretValueTypeSchema, - StoreSecretValueTypeSchema, - type StoreSecretValueType -} from '@fastgpt/global/common/secret/type'; +import type { StoreSecretValueType } from '@fastgpt/global/common/secret/type'; import { retryFn } from '@fastgpt/global/common/system/utils'; import { AppTypeEnum } from '@fastgpt/global/core/app/constants'; import { AppToolSourceEnum } from '@fastgpt/global/core/app/tool/constants'; -import { MongoApp } from './schema'; -import type { McpToolDataType } from '@fastgpt/global/core/app/tool/mcpTool/type'; import { UserError } from '@fastgpt/global/common/error/utils'; import $RefParser from '@apidevtools/json-schema-ref-parser'; import { getLogger, LogCategories } from '../../common/logger'; import { isInternalAddress, PRIVATE_URL_TEXT } from '../../common/system/utils'; import { decodeMcpToolSetNodesFromStorage } from './jsonSchemaStorage'; import { McpToolSetRuntimeConfigSchema } from '@fastgpt/global/core/workflow/type/node'; +import { getAppLatestVersion, type AppPublishedWorkflow } from './version/controller'; const logger = getLogger(LogCategories.MODULE.APP.MCP_TOOLS); @@ -423,46 +418,27 @@ export class MCPClient { } /** Read the current or legacy MCP child tools from a toolset app. */ -export const getMCPChildren = async (app: AppSchemaType): Promise => { +export const getMCPChildren = async ( + app: AppSchemaType, + workflow?: AppPublishedWorkflow +): Promise => { if (app.type !== AppTypeEnum.mcpToolSet) return []; + const nodes = decodeMcpToolSetNodesFromStorage( + (workflow ?? (await getAppLatestVersion(String(app._id), app))).nodes + ); + const node = nodes[0]; + if (!node) return []; + const id = String(app._id); - const modules = decodeMcpToolSetNodesFromStorage(app.modules); - const toolSet = McpToolSetRuntimeConfigSchema.safeParse(modules[0]?.toolConfig?.mcpToolSet).data; - - if (toolSet) { - return ( - toolSet.toolList.map((item) => ({ - ...item, - id: `${AppToolSourceEnum.mcp}-${id}/${item.name}`, - avatar: app.avatar - })) ?? [] - ); - } else { - // Old mcp toolset - const children = await MongoApp.find({ - teamId: app.teamId, - parentId: id - }).lean(); - - return children.map((item) => { - const node = item.modules[0]; - const toolData: McpToolDataType = node.inputs[0].value; - const { headerSecret, ...toolConfig } = toolData; - const normalizedHeaders = (() => { - if (!headerSecret) return undefined; - // 旧版本同时存在命名请求头映射和单个密钥;映射优先,避免重复包装 Authorization。 - const headerMap = StoreSecretValueTypeSchema.safeParse(headerSecret); - if (headerMap.success) return headerMap.data; - return { Authorization: SecretValueTypeSchema.parse(headerSecret) }; - })(); - - return { - avatar: app.avatar, - id: `${AppToolSourceEnum.mcp}-${id}/${item.name}`, - ...toolConfig, - ...(normalizedHeaders ? { headerSecret: normalizedHeaders } : {}) - }; - }); - } + + const toolSet = McpToolSetRuntimeConfigSchema.safeParse(node.toolConfig?.mcpToolSet).data; + + if (!toolSet) return []; + + return toolSet.toolList.map((item) => ({ + ...item, + id: `${AppToolSourceEnum.mcp}-${id}/${item.name}`, + avatar: app.avatar + })); }; diff --git a/packages/service/core/app/resourceLookup.ts b/packages/service/core/app/resourceLookup.ts new file mode 100644 index 000000000000..79d524fe5220 --- /dev/null +++ b/packages/service/core/app/resourceLookup.ts @@ -0,0 +1,79 @@ +import type { AppResourceType } from '@fastgpt/global/core/app/type'; +import { Types } from '../../common/mongo'; +import { MongoApp } from './schema'; +import { AppVersionCollectionName } from './version/schema'; +import { buildAppResourceMongoQuery } from './resources'; + +type PublishedAppResource = { type: AppResourceType; id: string }; +type MatchedPublishedApp = { + _id: unknown; + publishedResources?: PublishedAppResource[]; +}; + +/** + * 按当前正式 Version 反查引用了指定资源的团队 App。 + * 只查已有 publishedVersionId 的 App;4163 会给非文件夹 App 补齐该指针。 + * 先通过 $lookup 把资源匹配下推到 Mongo 拿到命中的 App id,再按 id 投影加载, + * 避免全团队 App 入内存,同时保留 find 的投影类型推断。 + */ +export const findTeamAppsByPublishedResource = async ({ + teamId, + type, + ids, + projection +}: { + teamId: string; + type: AppResourceType; + ids: string | string[]; + projection?: string; +}) => { + const idList = Array.isArray(ids) ? ids : [ids]; + const resourceQuery = buildAppResourceMongoQuery({ type, ids: idList }).resources; + // 聚合 $match 不做 mongoose 的 find 式自动转型,团队 id 需显式转 ObjectId。 + const teamObjectId = Types.ObjectId.isValid(teamId) ? new Types.ObjectId(teamId) : teamId; + + const matched = await MongoApp.aggregate([ + { + $match: { + teamId: teamObjectId, + deleteTime: null, + publishedVersionId: { $exists: true, $ne: null } + } + }, + { + $lookup: { + from: AppVersionCollectionName, + localField: 'publishedVersionId', + foreignField: '_id', + as: 'published' + } + }, + { $unwind: { path: '$published' } }, + { $match: { 'published.resources': resourceQuery } }, + { $project: { publishedResources: '$published.resources' } } + ]); + + const appIds: string[] = []; + const counts = new Map(); + matched.forEach((app) => { + appIds.push(String(app._id)); + const matchedResourceIds = new Set( + (app.publishedResources ?? []) + .filter((resource) => resource.type === type && idList.includes(resource.id)) + .map((resource) => resource.id) + ); + matchedResourceIds.forEach((resourceId) => { + counts.set(resourceId, (counts.get(resourceId) ?? 0) + 1); + }); + }); + + const apps = + appIds.length > 0 + ? await MongoApp.find( + { _id: { $in: appIds } }, + `_id publishedVersionId ${projection ?? ''}` + ).lean() + : []; + + return { apps, counts }; +}; diff --git a/packages/service/core/app/resourceRefs.ts b/packages/service/core/app/resourceRefs.ts deleted file mode 100644 index 439bc00469c5..000000000000 --- a/packages/service/core/app/resourceRefs.ts +++ /dev/null @@ -1,53 +0,0 @@ -import type { AppResourceRefsType } from '@fastgpt/global/core/app/type'; -import { NodeInputKeyEnum } from '@fastgpt/global/core/workflow/constants'; -import type { StoreNodeItemType } from '@fastgpt/global/core/workflow/type/node'; - -export const AppResourceRefsSkillIdsPath = 'resourceRefs.skillIds'; - -/** - * 从应用 workflow 节点中提取该版本引用的资源索引。 - * 当前只维护 skillIds,后续有真实消费场景时再补充其他资源字段。 - * 创建、保存版本、发布和历史回填共用同一套规则,避免不同写入路径统计口径漂移。 - */ -export function extractAppResourceRefsFromNodes( - nodes: StoreNodeItemType[] | null | undefined = [] -): AppResourceRefsType { - const skillIds = new Set(); - - const nodeList = Array.isArray(nodes) ? nodes : []; - - nodeList.forEach((node) => { - node.inputs?.forEach((input) => { - if (input.key !== NodeInputKeyEnum.skills) return; - - const value = input.value; - const skills = Array.isArray(value) ? value : value ? [value] : []; - - skills.forEach((item) => { - const skillId = item?.skillId; - if (skillId) { - skillIds.add(String(skillId)); - } - }); - }); - }); - - return { - skillIds: Array.from(skillIds) - }; -} - -const getSkillIdCondition = (skillIds: string | string[]) => { - const list = Array.isArray(skillIds) ? skillIds : [skillIds]; - return list.length === 1 ? list[0] : { $in: list }; -}; - -/** - * 构造基于 resourceRefs 的 Skill 引用查询。 - * apps.resourceRefs 是最新发布版本缓存,app_versions.resourceRefs 是版本事实记录。 - */ -export function buildAppSkillRefMongoQuery(skillIds: string | string[]) { - return { - [AppResourceRefsSkillIdsPath]: getSkillIdCondition(skillIds) - }; -} diff --git a/packages/service/core/app/resources.ts b/packages/service/core/app/resources.ts new file mode 100644 index 000000000000..90ce830886d4 --- /dev/null +++ b/packages/service/core/app/resources.ts @@ -0,0 +1,394 @@ +import { + AppResourcesSchema, + type AppChatConfigType, + type AppResource, + type AppResourceType, + type AppResourcesType +} from '@fastgpt/global/core/app/type'; +import { AppToolSourceEnum } from '@fastgpt/global/core/app/tool/constants'; +import { splitCombineToolId, splitToolsetToolPluginId } from '@fastgpt/global/core/app/tool/utils'; +import { NodeInputKeyEnum } from '@fastgpt/global/core/workflow/constants'; +import { FlowNodeTypeEnum } from '@fastgpt/global/core/workflow/node/constant'; +import type { StoreNodeItemType } from '@fastgpt/global/core/workflow/type/node'; +import type { RuntimeNodeItemType } from '@fastgpt/global/core/workflow/runtime/type'; +import { + isWorkflowSystemModelInput, + nodeInputIsReference +} from '@fastgpt/global/core/workflow/utils'; +import { getLogger, LogCategories } from '../../common/logger'; + +const resourceLogger = getLogger(LogCategories.MODULE.APP); + +type AppResourceModelType = Extract['data']['modelType']; + +const modelInputTypes = new Map([ + [NodeInputKeyEnum.aiModelId, 'llm'], + [NodeInputKeyEnum.datasetSearchRerankModelId, 'rerank'], + [NodeInputKeyEnum.datasetSearchExtensionModelId, 'llm'], + [NodeInputKeyEnum.datasetDeepSearchModelId, 'llm'] +]); + +const getValueList = (value: unknown) => + Array.isArray(value) ? value : value === undefined || value === null ? [] : [value]; + +const getObjectValue = (value: unknown, key: string) => { + if (!value || typeof value !== 'object' || Array.isArray(value)) return; + return (value as Record)[key]; +}; + +const getStringValue = (value: unknown) => { + if (typeof value === 'string' && value) return value; + if (typeof value === 'number') return String(value); + return; +}; + +/** + * 把节点/对话配置里的模型引用收成稳定标识。 + * 新写入只保留 modelId;解析失败时保留现场 ID,提取器只记录不鉴权。 + */ +const getModelId = (value: unknown, modelType: AppResourceModelType) => { + const rawValue = + getStringValue(value) ?? + getStringValue(getObjectValue(value, 'modelId')) ?? + getStringValue(getObjectValue(value, 'model')); + if (!rawValue || /^\{\{.*\}\}$/.test(rawValue)) return; + + const resolved = + global.systemModelMap?.get(`id:${rawValue}`) ?? global.systemModelMap?.get(`model:${rawValue}`); + if (resolved && resolved.type === modelType) return resolved.modelId; + return rawValue; +}; + +/** 资源去重键;模型额外包含 modelType,避免同名模型冲突。 */ +export const getAppResourceKey = (resource: AppResource) => + resource.type === 'model' + ? `${resource.type}:${resource.data.modelType}:${resource.id}` + : `${resource.type}:${resource.id}`; + +const isAclAppResource = (resource: AppResource) => + resource.type === 'agent' || + resource.type === 'tool' || + resource.type === 'dataset' || + resource.type === 'skill'; + +const isRecord = (value: unknown): value is Record => + typeof value === 'object' && value !== null && !Array.isArray(value); + +/** + * 从历史 `resourceRefs.skillIds` 取出有效 skill id。 + * 该字段在本次重构前已存在于 App / Version,4163 `$unset` 前读路径仍要 merge。 + */ +export const getLegacySkillIds = (resourceRefs: unknown): string[] => { + if (!isRecord(resourceRefs) || !Array.isArray(resourceRefs.skillIds)) return []; + return resourceRefs.skillIds.filter( + (id): id is string => typeof id === 'string' && id.length > 0 + ); +}; + +const normalizeToolNames = (toolNames?: string[]) => { + const names = Array.from(new Set(toolNames?.filter(Boolean) ?? [])).sort(); + return names.length ? names : undefined; +}; + +const normalizeAppResource = (resource: AppResource): AppResource => { + if (resource.type !== 'tool') return resource; + + // MCP/HTTP 节点保存的是 child tool id,但权限主体是 toolset app。 + // 快照落库和读取都统一到 parent id,避免同一资源在差量比较时被拆成两条。 + const normalizedTool = resource.id.includes('/') + ? normalizeAppToolResource(resource.id) + : undefined; + const toolNames = normalizeToolNames([ + ...(resource.data?.toolNames ?? []), + ...(normalizedTool?.toolName ? [normalizedTool.toolName] : []) + ]); + return { + type: 'tool', + id: normalizedTool?.id ?? resource.id, + ...(toolNames ? { data: { toolNames } } : {}) + }; +}; + +/** 合并并稳定化资源快照,供提取器和迁移脚本共用。 */ +export const mergeAppResources = (resources: AppResourcesType): AppResourcesType => { + const resourceMap = new Map(); + + resources.forEach((resource) => { + const normalizedResource = normalizeAppResource(resource); + const key = getAppResourceKey(normalizedResource); + const current = resourceMap.get(key); + if (!current) { + resourceMap.set(key, normalizedResource); + return; + } + + if (normalizedResource.type !== 'tool' || current.type !== 'tool') return; + if (!normalizedResource.data?.toolNames) { + resourceMap.set(key, { type: 'tool', id: normalizedResource.id }); + return; + } + if (!current.data?.toolNames) return; + + resourceMap.set(key, { + type: 'tool', + id: normalizedResource.id, + data: { + toolNames: Array.from( + new Set([...current.data.toolNames, ...normalizedResource.data.toolNames]) + ).sort() + } + }); + }); + + return Array.from(resourceMap.values()).sort((a, b) => { + const typeCompare = a.type.localeCompare(b.type); + if (typeCompare) return typeCompare; + + const idCompare = a.id.localeCompare(b.id); + if (idCompare) return idCompare; + + if (a.type === 'model' && b.type === 'model') { + return a.data.modelType.localeCompare(b.data.modelType); + } + + return 0; + }); +}; + +/** 判断节点是否通过工作流引用动态提供指定资源输入。 */ +export const nodeHasDynamicInput = ( + node: Pick | undefined, + keys: string[] +) => + node?.inputs?.some((input) => keys.includes(input.key) && nodeInputIsReference(input)) ?? false; + +/** 将工作流中的完整工具 ID 转为应用资源权限主体。 */ +export const normalizeAppToolResource = (toolId: unknown) => { + const id = getStringValue(toolId); + if (!id) return; + + try { + const { source, pluginId, authAppId } = splitCombineToolId(id); + if ( + source === AppToolSourceEnum.systemTool || + source === AppToolSourceEnum.commercial || + source === AppToolSourceEnum.community + ) { + return; + } + + const parentId = authAppId ?? pluginId; + if (!parentId) return; + + if (source === AppToolSourceEnum.mcp || source === AppToolSourceEnum.http) { + const { parentId: parsedParentId, toolName } = splitToolsetToolPluginId(pluginId); + return { + id: parsedParentId || parentId, + ...(toolName ? { toolName } : {}) + }; + } + + return { id: parentId }; + } catch { + return; + } +}; + +/** 从工作流节点和对话配置提取稳定、扁平的资源快照。 */ +export const extractAppResources = ({ + nodes = [], + chatConfig +}: { + nodes?: Array; + chatConfig?: AppChatConfigType; +}): AppResourcesType => { + const resources: AppResource[] = []; + const addResource = (resource: AppResource) => resources.push(resource); + + const addTool = (toolId: unknown) => { + const resource = normalizeAppToolResource(toolId); + if (!resource) return; + addResource({ + type: 'tool', + id: resource.id, + ...(resource.toolName ? { data: { toolNames: [resource.toolName] } } : {}) + }); + }; + + const addDataset = (value: unknown) => { + getValueList(value).forEach((item) => { + const id = + getStringValue(item) ?? + getStringValue(getObjectValue(item, 'datasetId')) ?? + getStringValue(getObjectValue(item, 'id')); + if (id) addResource({ type: 'dataset', id }); + const nested = getObjectValue(item, 'datasets'); + if (nested) addDataset(nested); + }); + }; + + const addSkills = (value: unknown) => { + getValueList(value).forEach((item) => { + const id = + getStringValue(item) ?? + getStringValue(getObjectValue(item, 'skillId')) ?? + getStringValue(getObjectValue(item, 'id')); + if (id) addResource({ type: 'skill', id }); + }); + }; + + const addModels = (value: unknown, modelType: AppResourceModelType) => { + getValueList(value).forEach((item) => { + const id = getModelId(item, modelType); + if (id) addResource({ type: 'model', id, data: { modelType } }); + }); + }; + + nodes.forEach((node) => { + const isStaticInputEnabled = (key: string) => + node.inputs?.some( + (item) => item.key === key && !nodeInputIsReference(item) && item.value === true + ) ?? false; + + if (node.flowNodeType === FlowNodeTypeEnum.appModule && node.pluginId) { + addResource({ type: 'agent', id: node.pluginId }); + } + if (node.flowNodeType === FlowNodeTypeEnum.pluginModule) addTool(node.pluginId); + if ( + (node.flowNodeType === FlowNodeTypeEnum.tool || + node.flowNodeType === FlowNodeTypeEnum.toolSet) && + node.pluginId + ) { + addTool(node.pluginId); + } + addTool(node.toolConfig?.mcpTool?.toolId); + addTool(node.toolConfig?.httpTool?.toolId); + + node.inputs?.forEach((input) => { + if (nodeInputIsReference(input)) return; + if (input.key === NodeInputKeyEnum.skills) addSkills(input.value); + if ( + input.key === NodeInputKeyEnum.datasetSelectList || + input.key === NodeInputKeyEnum.datasetParams + ) { + addDataset(input.value); + } + if (input.key === NodeInputKeyEnum.datasetParams && isRecord(input.value)) { + if (input.value[NodeInputKeyEnum.datasetSearchUsingReRank] === true) { + addModels(input.value[NodeInputKeyEnum.datasetSearchRerankModelId], 'rerank'); + } + if (input.value[NodeInputKeyEnum.datasetSearchUsingExtensionQuery] === true) { + addModels(input.value[NodeInputKeyEnum.datasetSearchExtensionModelId], 'llm'); + } + } + if (input.key === NodeInputKeyEnum.selectedTools) { + getValueList(input.value).forEach((tool) => { + const toolId = getObjectValue(tool, 'id') ?? getObjectValue(tool, 'toolId'); + addTool(toolId); + }); + } + if (input.key === NodeInputKeyEnum.runAppSelectApp) { + const id = + getStringValue(getObjectValue(input.value, 'appId')) ?? + getStringValue(getObjectValue(input.value, 'id')) ?? + getStringValue(input.value); + if (id) addResource({ type: 'agent', id }); + } + const modelType = modelInputTypes.get(input.key); + const enabled = + input.key === NodeInputKeyEnum.datasetSearchRerankModelId + ? isStaticInputEnabled(NodeInputKeyEnum.datasetSearchUsingReRank) + : input.key === NodeInputKeyEnum.datasetSearchExtensionModelId + ? isStaticInputEnabled(NodeInputKeyEnum.datasetSearchUsingExtensionQuery) + : input.key === NodeInputKeyEnum.datasetDeepSearchModelId + ? isStaticInputEnabled(NodeInputKeyEnum.datasetDeepSearch) + : true; + if (modelType && enabled && isWorkflowSystemModelInput({ node, input })) { + addModels(input.value, modelType); + } + }); + }); + + if (chatConfig?.questionGuide?.open) addModels(chatConfig.questionGuide.modelId, 'llm'); + if (chatConfig?.ttsConfig?.type === 'model') addModels(chatConfig.ttsConfig.modelId, 'tts'); + + return mergeAppResources(resources); +}; + +/** + * 解析一条 Version 上的 resources。 + * 数组(含空数组)视为已落库快照;缺字段则按 nodes 提取,并合并旧 resourceRefs.skillIds。 + */ +export const resolveStoredAppResources = ({ + resources, + nodes, + chatConfig, + resourceRefs +}: { + resources?: unknown; + nodes?: Array; + chatConfig?: AppChatConfigType; + resourceRefs?: unknown; +}): AppResourcesType => { + if (Array.isArray(resources)) { + const parsed = AppResourcesSchema.safeParse(resources); + if (parsed.success) return mergeAppResources(parsed.data); + resourceLogger.warn('Invalid stored app resources, fallback to node extraction'); + } + + const extracted = extractAppResources({ + nodes: nodes ?? [], + chatConfig + }); + return mergeAppResources([ + ...extracted, + ...getLegacySkillIds(resourceRefs).map((id) => ({ type: 'skill' as const, id })) + ]); +}; + +/** + * 相对上一版快照拆出无需重验的资源和新增 ACL 资源。 + * toolNames 随本次 extract 走,不单独作为权限增量。 + */ +export const splitExtractedAppResources = ({ + extracted, + baseline +}: { + extracted: AppResourcesType; + baseline: AppResourcesType; +}) => { + const baselineKeys = new Set( + baseline.filter(isAclAppResource).map((resource) => getAppResourceKey(resource)) + ); + const kept: AppResourcesType = []; + const added: AppResourcesType = []; + + extracted.forEach((resource) => { + if (!isAclAppResource(resource) || baselineKeys.has(getAppResourceKey(resource))) { + kept.push(resource); + return; + } + added.push(resource); + }); + + return { kept, added }; +}; + +/** 构造资源反查条件,type 和 id 必须通过 elemMatch 命中同一条资源。 */ +export const buildAppResourceMongoQuery = ({ + type, + ids +}: { + type: AppResourceType; + ids: string | string[]; +}) => { + const list = Array.isArray(ids) ? ids : [ids]; + return { + resources: { + $elemMatch: { + type, + id: list.length === 1 ? list[0] : { $in: list } + } + } + }; +}; diff --git a/packages/service/core/app/schema.ts b/packages/service/core/app/schema.ts index 43fa3d7438ed..63bfbedabffc 100644 --- a/packages/service/core/app/schema.ts +++ b/packages/service/core/app/schema.ts @@ -8,20 +8,6 @@ import { export const AppCollectionName = 'apps'; -export const chatConfigType = { - welcomeText: String, - welcomeConfig: Object, - variables: Array, - questionGuide: Object, - ttsConfig: Object, - whisperConfig: Object, - scheduledTriggerConfig: Object, - chatInputGuide: Object, - fileSelectConfig: Object, - instruction: String, - autoExecute: Object -}; - // schema const AppSchema = new Schema( { @@ -72,17 +58,25 @@ const AppSchema = new Schema( default: () => new Date() }, - // Workflow data + /** @deprecated 仅供 4.16.3 历史数据迁移读取,正常工作流使用 app_versions.nodes */ modules: { type: Array, - default: [] + default: undefined }, + /** @deprecated 仅供 4.16.3 历史数据迁移读取,正常工作流使用 app_versions.edges */ edges: { type: Array, - default: [] + default: undefined }, + /** @deprecated 仅供 4.16.3 历史数据迁移读取,正常工作流使用 app_versions.chatConfig */ chatConfig: { - type: chatConfigType + type: Object, + default: undefined + }, + /** @deprecated 仅供 4.16.3 资源快照迁移读取 skillIds */ + resourceRefs: { + type: Object, + default: undefined }, // Tool config @@ -109,11 +103,9 @@ const AppSchema = new Schema( scheduledTriggerNextTime: { type: Date }, - resourceRefs: { - skillIds: { - type: [String], - default: [] - } + publishedVersionId: { + type: Schema.Types.ObjectId, + ref: 'app_versions' }, inheritPermission: { type: Boolean, @@ -144,7 +136,11 @@ defineIndex(AppSchema, { key: { teamId: 1, createTime: 1 } }); defineIndex(AppSchema, { key: { teamId: 1, type: 1 } }); defineIndex(AppSchema, { key: { teamId: 1, parentId: 1 } }); defineIndex(AppSchema, { - key: { teamId: 1, deleteTime: 1, 'resourceRefs.skillIds': 1 } + key: { teamId: 1, deleteTime: 1, publishedVersionId: 1 } +}); +defineIndex(AppSchema, { + key: { teamId: 1, deleteTime: 1, 'resourceRefs.skillIds': 1 }, + deprecated: true }); // Schedule diff --git a/packages/service/core/app/tool/httpTool/entity.ts b/packages/service/core/app/tool/httpTool/entity.ts index 9256ad3efde7..886b9833dab6 100644 --- a/packages/service/core/app/tool/httpTool/entity.ts +++ b/packages/service/core/app/tool/httpTool/entity.ts @@ -1,8 +1,7 @@ -import type { AppSchemaType } from '@fastgpt/global/core/app/type'; import { MongoApp } from '../../schema'; -import { decodeHttpToolSetNodesFromStorage } from '../../jsonSchemaStorage'; +import type { AppSchemaType } from '@fastgpt/global/core/app/type'; -export const getHttpToolsets = ({ +export const getHttpToolsets = async ({ teamId, ids, field @@ -11,18 +10,9 @@ export const getHttpToolsets = ({ ids: string[]; field?: Record; }): Promise => { - return MongoApp.find({ teamId, _id: { $in: ids } }, field) - .lean() - .then((apps) => { - let changed = false; - const decodedApps = apps.map((app) => { - const modules = decodeHttpToolSetNodesFromStorage(app.modules); - if (modules === app.modules) return app; - - changed = true; - return { ...app, modules }; - }); - - return changed ? decodedApps : apps; - }); + const apps = await MongoApp.find( + { teamId, _id: { $in: ids } }, + field ? { ...field, publishedVersionId: true } : undefined + ).lean(); + return apps; }; diff --git a/packages/service/core/app/tool/mcpTool/entity.ts b/packages/service/core/app/tool/mcpTool/entity.ts index edb9caa65343..907034b7d0b4 100644 --- a/packages/service/core/app/tool/mcpTool/entity.ts +++ b/packages/service/core/app/tool/mcpTool/entity.ts @@ -1,8 +1,7 @@ -import type { AppSchemaType } from '@fastgpt/global/core/app/type'; import { MongoApp } from '../../schema'; -import { decodeMcpToolSetNodesFromStorage } from '../../jsonSchemaStorage'; +import type { AppSchemaType } from '@fastgpt/global/core/app/type'; -export const getMcpToolsets = ({ +export const getMcpToolsets = async ({ teamId, ids, field @@ -11,18 +10,9 @@ export const getMcpToolsets = ({ ids: string[]; field?: Record; }): Promise => { - return MongoApp.find({ teamId, _id: { $in: ids } }, field) - .lean() - .then((apps) => { - let changed = false; - const decodedApps = apps.map((app) => { - const modules = decodeMcpToolSetNodesFromStorage(app.modules); - if (modules === app.modules) return app; - - changed = true; - return { ...app, modules }; - }); - - return changed ? decodedApps : apps; - }); + const apps = await MongoApp.find( + { teamId, _id: { $in: ids } }, + field ? { ...field, publishedVersionId: true } : undefined + ).lean(); + return apps; }; diff --git a/packages/service/core/app/tool/utils/client.ts b/packages/service/core/app/tool/utils/client.ts index a6bfc6365d08..288970c627ee 100644 --- a/packages/service/core/app/tool/utils/client.ts +++ b/packages/service/core/app/tool/utils/client.ts @@ -49,7 +49,6 @@ import { import { Types } from 'mongoose'; import { getHTTPToolList } from '../../http'; import { getMCPChildren } from '../../mcp'; -import { decodeToolSetNodesFromStorage } from '../../jsonSchemaStorage'; import { MongoApp } from '../../schema'; import { getAppVersionById, checkIsLatestVersion } from '../../version/controller'; import { SystemToolRepo } from '../systemTool/systemTool.repo'; @@ -286,19 +285,11 @@ export async function getClientToolPreviewNode({ const isToolSetApp = item.type === AppTypeEnum.mcpToolSet || item.type === AppTypeEnum.httpToolSet; - const version = isToolSetApp - ? { - versionId: undefined, - versionName: undefined, - nodes: [...decodeToolSetNodesFromStorage(item.modules)], - edges: item.edges, - chatConfig: item.chatConfig - } - : await getAppVersionById({ - appId: pluginId, - versionId: versionId || undefined, - app: item - }); + const version = await getAppVersionById({ + appId: pluginId, + versionId: versionId || undefined, + app: item + }); const isLatest = !isToolSetApp && version.versionId && Types.ObjectId.isValid(version.versionId) @@ -308,28 +299,6 @@ export async function getClientToolPreviewNode({ }) : true; - // Adapt - if (item.type === AppTypeEnum.mcpToolSet && !version.nodes[0]?.toolConfig?.mcpToolSet) { - const children = await getMCPChildren(item); - version.nodes[0] = { - ...version.nodes[0], - // 仅在生成新预览时去掉已知旧配置槽,保留普通 IO;不能回写或截断存量节点输入。 - inputs: (version.nodes[0]?.inputs ?? []).filter( - (input) => - input.key !== NodeInputKeyEnum.toolSetData || - !input.renderTypeList.includes(FlowNodeInputTypeEnum.hidden) - ), - toolConfig: { - ...version.nodes[0]?.toolConfig, - mcpToolSet: { - toolList: children, - url: '', - headerSecret: {} - } - } - }; - } - const shouldReturnVersion = !isToolSetApp && (versionId ? true : versionId === undefined && getLatestVersion); diff --git a/packages/service/core/app/tool/workflowTool/service.ts b/packages/service/core/app/tool/workflowTool/service.ts index 5fce68501205..667f80a43529 100644 --- a/packages/service/core/app/tool/workflowTool/service.ts +++ b/packages/service/core/app/tool/workflowTool/service.ts @@ -10,7 +10,7 @@ const unsupportedInputMessage = '系统工具暂不支持关联包含文件、 /** * 校验工作流是否可以关联为系统工具。 - * 关联使用最新已发布版本;应用尚未发布时由 getAppLatestVersion 回退到应用当前草稿。 + * 关联使用最新已发布版本(publishedVersionId,否则最新 isPublish)。 * 内部变量会保留在 JSON Schema 中供 runtime 恢复默认值,并在模型与外部参数边界过滤, * 因此不属于这里的禁止类型。 */ diff --git a/packages/service/core/app/utils.ts b/packages/service/core/app/utils.ts index acb129ce18af..b288548b5063 100644 --- a/packages/service/core/app/utils.ts +++ b/packages/service/core/app/utils.ts @@ -18,6 +18,7 @@ import { getClientToolPreviewNode } from './tool/utils/client'; import { authAppByTmbId } from '../../support/permission/app/auth'; import { ReadPermissionVal } from '@fastgpt/global/support/permission/constant'; import { getErrText } from '@fastgpt/global/common/error/utils'; +import { AppErrEnum } from '@fastgpt/global/common/error/code/app'; import { isSystemOrCommercialToolId, splitCombineToolId @@ -33,10 +34,17 @@ import { type SelectedAgentSkillItemType } from '@fastgpt/global/core/app/formEdit/type'; import { authSkillByTmbId } from '../../support/permission/skill/auth'; +import { MongoAgentSkills } from '../ai/skill/model/schema'; +import { AgentSkillSourceEnum } from '@fastgpt/global/core/ai/skill/constants'; +import { SkillErrEnum } from '@fastgpt/global/common/error/code/skill'; +import { authDatasetByTmbId } from '../../support/permission/dataset/auth'; +import { DatasetErrEnum } from '@fastgpt/global/common/error/code/dataset'; +import { getAppResourceKey, normalizeAppToolResource } from './resources'; import type { FlowNodeInputItemType, SelectedDatasetType } from '@fastgpt/global/core/workflow/type/io'; +import type { AppResourcesType } from '@fastgpt/global/core/app/type'; import { formatToolInputSecrets } from './tool/secretConfig'; import z from 'zod'; @@ -50,58 +58,98 @@ export async function rewriteAppWorkflowToDetail({ teamId, isRoot, ownerTmbId, - lang + viewerTmbId, + lang, + resources = [] }: { nodes: DetailWorkflowNode[]; teamId: string; isRoot: boolean; ownerTmbId: string; + /** 当前请求操作者;仅对快照外新增资源做 UI 侧权限提示。 */ + viewerTmbId?: string; lang?: localeType; + resources?: AppResourcesType; }) { + // 快照内资源属于已确认的历史基线,不使用当前操作者或应用 owner 重新鉴权。 + const snapshotResourceKeys = new Set(resources.map(getAppResourceKey)); + const hasSnapshotResource = (type: 'agent' | 'tool' | 'dataset' | 'skill', id: string) => + snapshotResourceKeys.has(getAppResourceKey({ type, id })); type SelectedDatasetSnapshot = Pick & Partial; const defaultDeletedDatasetAvatar = DatasetTypeMap[DatasetTypeEnum.dataset].avatar; + const authSnapshotExternalTool = async ({ + id, + resourceType + }: { + id: string; + resourceType: 'agent' | 'tool'; + }) => { + const normalizedResource = normalizeAppToolResource(id); + const resourceInSnapshot = + !!normalizedResource && hasSnapshotResource(resourceType, normalizedResource.id); + if ((!viewerTmbId || resourceInSnapshot) && !isRoot) return false; + + let authAppId: string | undefined; + try { + authAppId = splitCombineToolId(id).authAppId; + } catch { + return false; + } + if (!authAppId) return false; + + try { + await authAppByTmbId({ + tmbId: viewerTmbId ?? ownerTmbId, + appId: authAppId, + per: ReadPermissionVal, + isRoot + }); + return false; + } catch { + return true; + } + }; + const loadToolNode = async ({ id, versionId, - source + source, + resourceType = 'tool' }: { id: string; versionId?: string; source?: string; + resourceType?: 'agent' | 'tool'; }) => { - const { authAppId } = splitCombineToolId(id); + if (await authSnapshotExternalTool({ id, resourceType })) { + return { + success: false, + error: AppErrEnum.unAuthApp, + permissionDenied: true + }; + } try { - const [preview] = await Promise.all([ - getClientToolPreviewNode({ - appId: id, - versionId, - lang, - source, - teamId - }), - ...(authAppId - ? [ - authAppByTmbId({ - tmbId: ownerTmbId, - appId: authAppId, - per: ReadPermissionVal, - isRoot - }) - ] - : []) - ]); + const preview = await getClientToolPreviewNode({ + appId: id, + versionId, + lang, + source, + teamId + }); return { success: true, - data: preview + data: preview, + permissionDenied: false }; } catch (error) { return { success: false, - error: getErrText(error, '', lang) + error: getErrText(error, '', lang), + permissionDenied: false }; } }; @@ -113,28 +161,44 @@ export async function rewriteAppWorkflowToDetail({ const loadAgentSkill = async ( selectedSkill: AgentSkillSnapshot ): Promise => { + const skillId = String(selectedSkill.skillId); + const resourceInSnapshot = hasSnapshotResource('skill', skillId); try { - const { skill } = await authSkillByTmbId({ - tmbId: ownerTmbId, - skillId: selectedSkill.skillId, - per: ReadPermissionVal, - isRoot - }); + const skill = + viewerTmbId && !resourceInSnapshot + ? ( + await authSkillByTmbId({ + tmbId: viewerTmbId, + skillId, + per: ReadPermissionVal, + isRoot + }) + ).skill + : await MongoAgentSkills.findOne({ + _id: skillId, + deleteTime: null, + ...(isRoot ? {} : { $or: [{ teamId }, { source: AgentSkillSourceEnum.system }] }) + }).lean(); + + if (!skill) throw SkillErrEnum.unExist; return { skillId: String(skill._id), name: skill.name, description: skill.description, avatar: skill.avatar, - isDeleted: false + isDeleted: false, + permissionDenied: false }; - } catch { + } catch (error) { + const permissionDenied = error === SkillErrEnum.unAuthSkill; return { skillId: selectedSkill.skillId, name: selectedSkill.name ?? 'Invalid', description: selectedSkill.description ?? '', avatar: selectedSkill.avatar, - isDeleted: true + isDeleted: !permissionDenied, + permissionDenied }; } }; @@ -176,10 +240,28 @@ export async function rewriteAppWorkflowToDetail({ snapshot: SelectedDatasetSnapshot ): Promise => { const datasetId = String(snapshot.datasetId); - const dataset = await MongoDataset.findOne({ - _id: datasetId, - ...(!isRoot && teamId && { teamId }) - }).lean(); + const resourceInSnapshot = hasSnapshotResource('dataset', datasetId); + let dataset; + let permissionDenied = false; + + try { + dataset = + viewerTmbId && !resourceInSnapshot + ? ( + await authDatasetByTmbId({ + tmbId: viewerTmbId, + datasetId, + per: ReadPermissionVal, + isRoot + }) + ).dataset + : await MongoDataset.findOne({ + _id: datasetId, + ...(!isRoot && teamId && { teamId }) + }).lean(); + } catch (error) { + permissionDenied = error === DatasetErrEnum.unAuthDataset; + } if (dataset && !dataset.deleteTime) { return { @@ -187,7 +269,8 @@ export async function rewriteAppWorkflowToDetail({ avatar: dataset.avatar, name: dataset.name, vectorModel: getDatasetEmbeddingModel(dataset), - isDeleted: false + isDeleted: false, + permissionDenied }; } @@ -197,7 +280,8 @@ export async function rewriteAppWorkflowToDetail({ avatar: defaultDeletedDatasetAvatar, name: snapshot.name || '', vectorModel: snapshot.vectorModel || desensitizeSystemModel(getDefaultEmbeddingModelData()), - isDeleted: true + isDeleted: !permissionDenied, + ...(permissionDenied ? { permissionDenied: true } : {}) }; }; @@ -229,6 +313,7 @@ export async function rewriteAppWorkflowToDetail({ const result = await loadToolNode({ id: toolId, versionId: node.version ?? '', + resourceType: node.flowNodeType === FlowNodeTypeEnum.appModule ? 'agent' : 'tool', source: node.source ?? node.toolConfig?.systemTool?.source ?? @@ -246,7 +331,8 @@ export async function rewriteAppWorkflowToDetail({ diagram: preview.diagram, userGuide: preview.userGuide, courseUrl: preview.courseUrl, - readmeUrl: preview.readmeUrl + readmeUrl: preview.readmeUrl, + ...(result.permissionDenied ? { permissionDenied: true } : {}) }; node.versionLabel = preview.versionLabel; node.isLatestVersion = preview.isLatestVersion; @@ -281,7 +367,8 @@ export async function rewriteAppWorkflowToDetail({ } } else { node.pluginData = { - error: result.error + error: result.error, + ...(result.permissionDenied ? { permissionDenied: true } : {}) }; } } @@ -309,7 +396,8 @@ export async function rewriteAppWorkflowToDetail({ const result = await loadToolNode({ id: tool.id, versionId: tool.version, - source: tool.source + source: tool.source, + resourceType: 'tool' }); if (result.success) { const data = result.data!; @@ -357,6 +445,9 @@ export async function rewriteAppWorkflowToDetail({ (tool.toolConfig?.httpToolSet && 'toolId' in tool.toolConfig.httpToolSet) ? data.toolConfig : (tool.toolConfig ?? data.toolConfig), + ...(result.permissionDenied + ? { pluginData: { ...data.pluginData, permissionDenied: true } } + : {}), inputs: mergedInputs }; } else { @@ -379,7 +470,8 @@ export async function rewriteAppWorkflowToDetail({ outputs: [], configStatus: 'invalid' as const, pluginData: { - error: result.error + error: result.error, + ...(result.permissionDenied ? { permissionDenied: true } : {}) } }; } diff --git a/packages/service/core/app/version/controller.ts b/packages/service/core/app/version/controller.ts index 7797c09d0672..0c6e85836cb1 100644 --- a/packages/service/core/app/version/controller.ts +++ b/packages/service/core/app/version/controller.ts @@ -1,48 +1,317 @@ -import { type AppSchemaType } from '@fastgpt/global/core/app/type'; +import { + AppResourcesSchema, + type AppResourcesType, + type AppSchemaType +} from '@fastgpt/global/core/app/type'; import { MongoApp } from '../schema'; import { MongoAppVersion } from './schema'; -import { Types } from '../../../common/mongo'; +import { Types, type ClientSession } from '../../../common/mongo'; import { migrateWorkflowToCurrent } from '@fastgpt/global/core/workflow/migration'; import { decodeToolSetNodesFromStorage } from '../jsonSchemaStorage'; +import { resolveStoredAppResources } from '../resources'; +import type { AppVersionSchemaType } from '@fastgpt/global/core/app/version/type'; +import { AppErrEnum } from '@fastgpt/global/common/error/code/app'; +import { MongoTransactionConflictError } from '../../../common/mongo/sessionRun'; -export const getAppLatestVersion = async (appId: string, app?: AppSchemaType) => { - const migrationApp = - app ?? ((await MongoApp.findById(appId).lean()) as AppSchemaType | null | undefined); - const version = await MongoAppVersion.findOne({ - appId, - isPublish: true - }) - .sort({ - time: -1 - }) - .lean(); - - if (version) { - // 历史版本只迁移该版本自身的系统配置节点,不继承当前应用 chatConfig, - // 避免当前配置占位导致该版本中的欢迎语、定时任务等旧值被丢弃。 - const normalizedWorkflow = migrateWorkflowToCurrent({ - nodes: decodeToolSetNodesFromStorage(version.nodes), - edges: version.edges, - chatConfig: version.chatConfig - }); - return { - versionId: String(version._id), - versionName: version.versionName, - ...normalizedWorkflow - }; - } +type VersionResourceSource = Pick & { + resourceRefs?: unknown; +}; +type NormalizedWorkflow = ReturnType; +export type AppVersionWorkflow = NormalizedWorkflow & { + versionId?: string; + versionName?: string; + resources: AppResourcesType; +}; +export type AppPublishedWorkflow = Pick; + +const getVersionResourceSnapshot = ( + version: VersionResourceSource, + nodes = version.nodes, + chatConfig = version.chatConfig +): AppResourcesType => + resolveStoredAppResources({ + resources: version.resources, + nodes, + chatConfig, + resourceRefs: version.resourceRefs + }); + +const normalizeAppVersionWorkflow = (version: AppVersionSchemaType): AppVersionWorkflow => { + // 历史版本只迁移该版本自身的系统配置节点,不继承当前应用 chatConfig, + // 避免当前配置占位导致该版本中的欢迎语、定时任务等旧值被丢弃。 const normalizedWorkflow = migrateWorkflowToCurrent({ - nodes: decodeToolSetNodesFromStorage(migrationApp?.modules ?? []), - edges: migrationApp?.edges ?? [], - chatConfig: migrationApp?.chatConfig + nodes: decodeToolSetNodesFromStorage(version.nodes), + edges: version.edges, + chatConfig: version.chatConfig }); return { - versionId: migrationApp?.pluginData?.nodeVersion, - versionName: migrationApp?.name, + versionId: String(version._id), + versionName: version.versionName, + resources: getVersionResourceSnapshot( + version, + normalizedWorkflow.nodes, + normalizedWorkflow.chatConfig + ), ...normalizedWorkflow }; }; +const emptyVersionWorkflow = (): AppVersionWorkflow => { + const normalizedWorkflow = migrateWorkflowToCurrent({ + nodes: [], + edges: [], + chatConfig: undefined + }); + return { + versionId: undefined, + versionName: undefined, + resources: [] as AppResourcesType, + ...normalizedWorkflow + }; +}; + +export type AppVersionLookupApp = AppSchemaType; + +const loadApp = async (appId: string, app?: AppVersionLookupApp) => + app ?? ((await MongoApp.findById(appId).lean()) as AppSchemaType | null | undefined); + +/** + * 读取当前正式工作流:优先 publishedVersionId,否则最新 isPublish Version。 + * 找不到 Version 时返回空图,不再读 App.modules。 + */ +export const getAppLatestVersion = async (appId: string, app?: AppVersionLookupApp) => { + const migrationApp = await loadApp(appId, app); + const publishedVersion = + migrationApp?.publishedVersionId && + Types.ObjectId.isValid(String(migrationApp.publishedVersionId)) + ? await MongoAppVersion.findOne({ + _id: migrationApp.publishedVersionId, + appId + }).lean() + : null; + const version = + publishedVersion ?? + (await MongoAppVersion.findOne({ + appId, + isPublish: true + }) + .sort({ + time: -1, + _id: -1 + }) + .lean()); + + if (version) return normalizeAppVersionWorkflow(version); + return emptyVersionWorkflow(); +}; + +/** + * 读取编辑器工作副本:始终取该 App 最新写入的 Version,包含自动保存记录。 + * 找不到 Version 时返回空图,不再读 App.modules。 + */ +export const getAppDraftWorkflow = async (appId: string) => { + const draft = await getAppDraftVersion(appId); + if (draft) return normalizeAppVersionWorkflow(draft); + return emptyVersionWorkflow(); +}; + +/** + * 读取当前最新 Version,作为保存增量鉴权的 baseline。 + */ +export const getAppDraftVersion = async (appId: string, session?: ClientSession) => { + const query = MongoAppVersion.findOne({ appId }).sort({ time: -1, _id: -1 }); + if (session) query.session(session); + return query.lean(); +}; + +/** + * 读取当前草稿 Version 的资源快照,供保存增量鉴权做 baseline。 + * 没有草稿时返回空数组,本次提取全部视为新增。传入 session 时,读取会参与同一事务, + * 事务重试后也会重新读取最新的草稿 Version。 + */ +export const getAppDraftResourceBaseline = async (appId: string, session?: ClientSession) => { + const draft = await getAppDraftVersion(appId, session); + if (!draft) return []; + + // 非法快照不能回退为当前节点提取,否则曾被保存过滤掉的资源会重新变成 baseline, + // 从而绕过下一次保存/发布的新增资源鉴权。缺失字段仍按历史版本兼容逻辑提取。 + if (Array.isArray(draft.resources) && !AppResourcesSchema.safeParse(draft.resources).success) { + return []; + } + + return resolveStoredAppResources({ + resources: draft.resources, + nodes: decodeToolSetNodesFromStorage(draft.nodes), + chatConfig: draft.chatConfig, + resourceRefs: (draft as { resourceRefs?: unknown }).resourceRefs + }); +}; + +/** + * 在同一事务内更新当前正式 Version,并用 App 正式版本指针做 CAS。 + * ToolSet 没有独立草稿生命周期;无有效指针时只回退到最新正式版本,并在成功后补齐指针。 + * 并发发布改变指针时更新条件不再命中,交给事务入口重试,避免把工具配置写入旧版本。 + */ +export const updateAppPublishedVersion = async ({ + appId, + nodes, + resources, + session +}: { + appId: string; + nodes: AppVersionSchemaType['nodes']; + resources: AppResourcesType; + session: ClientSession; +}) => { + const app = await MongoApp.findById(appId, 'publishedVersionId').session(session).lean(); + if (!app) throw AppErrEnum.unExist; + + const pointerVersion = + app.publishedVersionId && Types.ObjectId.isValid(String(app.publishedVersionId)) + ? await MongoAppVersion.findOne( + { + _id: app.publishedVersionId, + appId + }, + '_id' + ) + .session(session) + .lean() + : null; + const version = + pointerVersion ?? + (await MongoAppVersion.findOne( + { + appId, + isPublish: true + }, + '_id' + ) + .sort({ time: -1, _id: -1 }) + .session(session) + .lean()); + + if (!version) throw AppErrEnum.unExist; + + const versionUpdateResult = await MongoAppVersion.updateOne( + { + _id: version._id, + appId + }, + { + $set: { + nodes, + resources + } + }, + { session } + ); + if (versionUpdateResult.matchedCount !== 1) { + throw new MongoTransactionConflictError( + new Error('Published app version changed during tool set update') + ); + } + + const expectedPublishedVersionId = + app.publishedVersionId && Types.ObjectId.isValid(String(app.publishedVersionId)) + ? app.publishedVersionId + : undefined; + const appFilter = expectedPublishedVersionId + ? { + _id: appId, + publishedVersionId: expectedPublishedVersionId + } + : { + _id: appId, + $or: [{ publishedVersionId: null }, { publishedVersionId: { $exists: false } }] + }; + const shouldRepairPublishedVersion = !pointerVersion; + const appUpdateResult = await MongoApp.updateOne( + appFilter, + { + $set: { + updateTime: new Date(), + ...(shouldRepairPublishedVersion ? { publishedVersionId: version._id } : {}) + } + }, + { session } + ); + if (appUpdateResult.matchedCount !== 1) { + throw new MongoTransactionConflictError( + new Error('Published app version pointer changed during tool set update') + ); + } + + return version._id; +}; + +/** 批量读取 App 当前正式 Version,供运行时入口复用同一份版本选择逻辑。 */ +export const getAppPublishedWorkflowMap = async ( + apps: AppSchemaType[] +): Promise> => { + if (apps.length === 0) return new Map(); + + const pointerIds = apps + .map((app) => app.publishedVersionId) + .filter((id): id is NonNullable => !!id && Types.ObjectId.isValid(String(id))); + + const versionById = new Map(); + const versionByAppId = new Map(); + + if (pointerIds.length > 0) { + const versions = await MongoAppVersion.find({ _id: { $in: pointerIds } }).lean(); + versions.forEach((version) => versionById.set(String(version._id), version)); + } + + const isPointerVersionForApp = ( + app: AppSchemaType, + version?: AppVersionSchemaType + ): version is AppVersionSchemaType => !!version && String(version.appId) === String(app._id); + + const appsNeedingLatestPublish = apps + .filter((app) => { + if (!app.publishedVersionId || !Types.ObjectId.isValid(String(app.publishedVersionId))) { + return true; + } + return !isPointerVersionForApp(app, versionById.get(String(app.publishedVersionId))); + }) + .map((app) => app._id); + + if (appsNeedingLatestPublish.length > 0) { + const latestPublished = await MongoAppVersion.aggregate<{ + _id: unknown; + doc: AppVersionSchemaType; + }>([ + { + $match: { + appId: { $in: appsNeedingLatestPublish }, + isPublish: true + } + }, + { $sort: { time: -1, _id: -1 } }, + { + $group: { + _id: '$appId', + doc: { $first: '$$ROOT' } + } + } + ]); + latestPublished.forEach((item) => versionByAppId.set(String(item._id), item.doc)); + } + + return new Map( + apps.map((app) => { + const pointerVersion = app.publishedVersionId + ? versionById.get(String(app.publishedVersionId)) + : undefined; + const version = isPointerVersionForApp(app, pointerVersion) + ? pointerVersion + : versionByAppId.get(String(app._id)); + return [String(app._id), { nodes: decodeToolSetNodesFromStorage(version?.nodes ?? []) }]; + }) + ); +}; + export const getAppVersionById = async ({ appId, versionId, @@ -50,30 +319,17 @@ export const getAppVersionById = async ({ }: { appId: string; versionId?: string; - app?: AppSchemaType; + app?: AppVersionLookupApp; }) => { - // 检查 versionId 是否符合 ObjectId 格式 if (versionId && Types.ObjectId.isValid(versionId)) { const version = await MongoAppVersion.findOne({ _id: versionId, appId }).lean(); - if (version) { - const normalizedWorkflow = migrateWorkflowToCurrent({ - nodes: decodeToolSetNodesFromStorage(version.nodes), - edges: version.edges, - chatConfig: version.chatConfig - }); - return { - versionId: String(version._id), - versionName: version.versionName, - ...normalizedWorkflow - }; - } + if (version) return normalizeAppVersionWorkflow(version); } - // If the version does not exist, the latest version is returned return getAppLatestVersion(appId, app); }; diff --git a/packages/service/core/app/version/schema.ts b/packages/service/core/app/version/schema.ts index 2f98a6f08d5d..47d8bc6b8763 100644 --- a/packages/service/core/app/version/schema.ts +++ b/packages/service/core/app/version/schema.ts @@ -1,11 +1,25 @@ import { defineIndex, connectionMongo, getMongoModel } from '../../../common/mongo'; const { Schema } = connectionMongo; import { type AppVersionSchemaType } from '@fastgpt/global/core/app/version/type'; -import { AppCollectionName, chatConfigType } from '../schema'; +import { AppCollectionName } from '../schema'; import { TeamMemberCollectionName } from '@fastgpt/global/support/user/team/constant'; export const AppVersionCollectionName = 'app_versions'; +const chatConfigType = { + welcomeText: String, + welcomeConfig: Object, + variables: Array, + questionGuide: Object, + ttsConfig: Object, + whisperConfig: Object, + scheduledTriggerConfig: Object, + chatInputGuide: Object, + fileSelectConfig: Object, + instruction: String, + autoExecute: Object +}; + const AppVersionSchema = new Schema( { tmbId: { @@ -36,11 +50,13 @@ const AppVersionSchema = new Schema( isPublish: Boolean, isAutoSave: Boolean, versionName: String, + resources: { + type: Array + }, + /** @deprecated 仅供 4.16.3 资源快照迁移读取 skillIds */ resourceRefs: { - skillIds: { - type: [String], - default: [] - } + type: Object, + default: undefined } }, { @@ -49,6 +65,9 @@ const AppVersionSchema = new Schema( ); defineIndex(AppVersionSchema, { key: { appId: 1, time: -1 } }); +defineIndex(AppVersionSchema, { + key: { appId: 1, 'resources.type': 1, 'resources.id': 1 } +}); export const MongoAppVersion = getMongoModel( AppVersionCollectionName, diff --git a/packages/service/core/workflow/dispatch/abandoned/runApp.ts b/packages/service/core/workflow/dispatch/abandoned/runApp.ts index 3774eea3a5a6..99d325f8fa7f 100644 --- a/packages/service/core/workflow/dispatch/abandoned/runApp.ts +++ b/packages/service/core/workflow/dispatch/abandoned/runApp.ts @@ -10,16 +10,17 @@ import { storeEdges2RuntimeEdges, storeNodes2RuntimeNodes } from '@fastgpt/global/core/workflow/runtime/utils'; -import type { NodeInputKeyEnum, NodeOutputKeyEnum } from '@fastgpt/global/core/workflow/constants'; +import { NodeInputKeyEnum, type NodeOutputKeyEnum } from '@fastgpt/global/core/workflow/constants'; import { DispatchNodeResponseKeyEnum } from '@fastgpt/global/core/workflow/runtime/constants'; import { getHistories, safePoints } from '../utils'; import { getWorkflowFileVariableInputs, WorkflowVariableState } from '../utils/variables'; import { chatValue2RuntimePrompt, runtimePrompt2ChatsValue } from '@fastgpt/global/core/chat/adapt'; import type { DispatchNodeResultType, ModuleDispatchProps } from '../../types/runtime'; -import { authAppByTmbId } from '../../../../support/permission/app/auth'; -import { ReadPermissionVal } from '@fastgpt/global/support/permission/constant'; import { getUserChatInfo } from '../../../../support/user/team/utils'; import { runWithDerivedWorkflowFileContext } from '../../utils/context'; +import { createWorkflowChildResourceContext, loadWorkflowAppResource } from '../../utils/resource'; +import { getAppVersionById } from '../../../app/version/controller'; +import { nodeHasDynamicInput } from '../../../app/resources'; type Props = ModuleDispatchProps<{ [NodeInputKeyEnum.userChatInput]: string; @@ -45,12 +46,21 @@ export const dispatchAppRequest = async (props: Props): Promise => { return Promise.reject('Input is empty'); } - // 检查该工作流的tmb是否有调用该app的权限(不是校验对话的人,是否有权限) - const { app: appData } = await authAppByTmbId({ + const dynamicApp = nodeHasDynamicInput(props.node, [NodeInputKeyEnum.runAppSelectApp]); + const appData = await loadWorkflowAppResource({ appId: app.id, - tmbId: runningAppInfo.tmbId, - per: ReadPermissionVal + tmbId: props.runningUserInfo.tmbId, + type: 'agent', + dynamic: dynamicApp }); + const childVersion = await getAppVersionById({ + appId: app.id, + app: appData + }); + const resourceContext = await createWorkflowChildResourceContext( + childVersion.resources, + String(appData.teamId) + ); workflowStreamResponse?.(workflowSseEvent.fastAnswerDelta('\n')); @@ -78,9 +88,10 @@ export const dispatchAppRequest = async (props: Props): Promise => { query: childQuery, histories: chatHistories, files: getWorkflowFileVariableInputs({ - variablesConfig: appData.chatConfig?.variables, + variablesConfig: childVersion.chatConfig.variables, inputVariables: childInputVariables }), + resourceContext, fn: async ({ resolveInputFile, query: filteredQuery, histories: filteredHistories }) => { filteredChildHistories = filteredHistories; filteredChildQuery = filteredQuery; @@ -91,7 +102,7 @@ export const dispatchAppRequest = async (props: Props): Promise => { chatId: props.chatId, responseChatItemId: props.responseChatItemId, histories: filteredHistories, - variablesConfig: appData.chatConfig?.variables, + variablesConfig: childVersion.chatConfig.variables, inputVariables: childInputVariables, externalVariables: externalProvider?.externalWorkflowVariables, sourceVariableState: variableState, @@ -102,12 +113,12 @@ export const dispatchAppRequest = async (props: Props): Promise => { ...props, runningAppInfo: childRunningAppInfo, runtimeNodes: storeNodes2RuntimeNodes( - appData.modules, - getWorkflowEntryNodeIds(appData.modules) + childVersion.nodes, + getWorkflowEntryNodeIds(childVersion.nodes) ), - runtimeEdges: storeEdges2RuntimeEdges(appData.edges), + runtimeEdges: storeEdges2RuntimeEdges(childVersion.edges), variableState: childVariableState, - chatConfig: appData.chatConfig, + chatConfig: childVersion.chatConfig, histories: filteredHistories, query: filteredQuery }); diff --git a/packages/service/core/workflow/dispatch/ai/agent/adapter/runtime.ts b/packages/service/core/workflow/dispatch/ai/agent/adapter/runtime.ts index b2bbb3456fc7..3043601abc64 100644 --- a/packages/service/core/workflow/dispatch/ai/agent/adapter/runtime.ts +++ b/packages/service/core/workflow/dispatch/ai/agent/adapter/runtime.ts @@ -24,6 +24,7 @@ type WorkflowAgentLoopRuntimeContext = ToolDispatchContext & { }; currentFiles: UseUserContextResult['currentFiles']; sandboxClient?: SandboxClient; + dynamicDataset?: boolean; }; type WorkflowAgentLoopRuntimeArtifacts = { diff --git a/packages/service/core/workflow/dispatch/ai/agent/adapter/userContext.ts b/packages/service/core/workflow/dispatch/ai/agent/adapter/userContext.ts index 8e653ffb2299..7bd8a07261cc 100644 --- a/packages/service/core/workflow/dispatch/ai/agent/adapter/userContext.ts +++ b/packages/service/core/workflow/dispatch/ai/agent/adapter/userContext.ts @@ -5,6 +5,11 @@ import type { ChatItemMiniType } from '@fastgpt/global/core/chat/type'; import { getAgentLoopHistories } from '../../../utils'; import { MongoDataset } from '../../../../../dataset/schema'; import { filterDatasetsByTmbId } from '../../../../../dataset/utils'; +import { + assertWorkflowDatasetResources, + getWorkflowDatasetResource +} from '../../../../utils/resource'; +import { getWorkflowResourceContext } from '../../../../utils/context'; import type { DeployedSkillInfo } from '../../../../../ai/sandbox/interface/runtime'; import { buildWorkflowAICurrentInputFiles, @@ -29,27 +34,36 @@ type AgentSelectedDatasetContext = AgentLoopCoreSelectedDatasetContext; export const loadAgentDatasetContext = async ( selectedDataset: AgentSelectedDatasetInput[] = [], tmbId: string, - authTmbId = false + authTmbId = false, + dynamicDataset = false ): Promise => { if (selectedDataset.length === 0) return []; const datasetIds = selectedDataset.map((item) => item.datasetId); - const authorizedDatasetIds = authTmbId - ? await filterDatasetsByTmbId({ - datasetIds, - tmbId - }) - : datasetIds; + assertWorkflowDatasetResources({ datasetIds, dynamic: dynamicDataset }); + const authorizedDatasetIds = + authTmbId || dynamicDataset + ? await filterDatasetsByTmbId({ + datasetIds, + tmbId + }) + : datasetIds; if (authorizedDatasetIds.length === 0) return []; - const datasets = await MongoDataset.find( - { - _id: { - $in: authorizedDatasetIds - } - }, - 'name intro' - ).lean(); + const resourceContext = getWorkflowResourceContext(); + const datasets = + resourceContext && !dynamicDataset + ? authorizedDatasetIds + .map((datasetId) => getWorkflowDatasetResource(datasetId)) + .filter((dataset): dataset is NonNullable => !!dataset) + : await MongoDataset.find( + { + _id: { + $in: authorizedDatasetIds + } + }, + 'name intro' + ).lean(); const datasetMap = new Map( datasets.map((item) => [ String(item._id), @@ -105,6 +119,7 @@ export const useUserContext = async ({ parseHistoryFiles = false, selectedDataset, authTmbId, + dynamicDataset = false, tmbId, timezone }: { @@ -118,6 +133,7 @@ export const useUserContext = async ({ parseHistoryFiles?: boolean; selectedDataset?: AgentSelectedDatasetInput[]; authTmbId?: boolean; + dynamicDataset?: boolean; tmbId: string; timezone: string; }): Promise => { @@ -151,7 +167,12 @@ export const useUserContext = async ({ }); // 获取知识库 - const selectedDatasetWithIntro = await loadAgentDatasetContext(selectedDataset, tmbId, authTmbId); + const selectedDatasetWithIntro = await loadAgentDatasetContext( + selectedDataset, + tmbId, + authTmbId, + dynamicDataset + ); return { chatHistories, diff --git a/packages/service/core/workflow/dispatch/ai/agent/index.ts b/packages/service/core/workflow/dispatch/ai/agent/index.ts index 631961f082f4..26d2424d2abe 100644 --- a/packages/service/core/workflow/dispatch/ai/agent/index.ts +++ b/packages/service/core/workflow/dispatch/ai/agent/index.ts @@ -42,6 +42,7 @@ import { runAgentLoopCoreWithSummary } from '../agentLoopCore/interface'; import { buildDefaultAgentSystemPrompt } from '../../../../ai/llm/agentLoop/interface'; +import { nodeHasDynamicInput } from '../../../../app/resources'; export type DispatchAgentModuleProps = ModuleDispatchProps<{ [NodeInputKeyEnum.history]?: ChatItemMiniType[]; @@ -136,6 +137,14 @@ export const dispatchRunAgent = async (props: DispatchAgentModuleProps): Promise } } = props; const datasetParams = getAgentDatasetParams(props.params); + const dynamicDataset = nodeHasDynamicInput(props.node, [ + NodeInputKeyEnum.datasetSelectList, + NodeInputKeyEnum.datasetParams + ]); + const dynamicSkills = + runningAppInfo.sourceType === ChatSourceTypeEnum.skillEdit || + nodeHasDynamicInput(props.node, [NodeInputKeyEnum.skills]); + const dynamicTools = nodeHasDynamicInput(props.node, [NodeInputKeyEnum.selectedTools]); const agentModel = getLLMModelData({ modelId, model }); // 旧 params.model 仅保留给兼容读取;Agent 请求链使用规范化 modelData。 props.params.model = agentModel.model; @@ -188,6 +197,7 @@ export const dispatchRunAgent = async (props: DispatchAgentModuleProps): Promise parseHistoryFiles, selectedDataset: datasetParams?.datasets, authTmbId: datasetParams?.authTmbId, + dynamicDataset, tmbId: runningUserInfo.tmbId, timezone, maxFileAmount: getWorkflowFileMaxAmount() @@ -203,18 +213,18 @@ export const dispatchRunAgent = async (props: DispatchAgentModuleProps): Promise }); } - // 初始化 sandbox:初始化、注入 skills、files - // skill 鉴权以应用 owner 为准,发布应用被他人调用时不因调用者缺少 skill 权限而跳过注入。 + // 初始化 sandbox:初始化、注入 skills、files。静态 Skill 由 Version 快照授权,动态 Skill 才检查运行人。 const { sandboxClient, currentWorkingDirectory, skillInfos } = await ensureAgentSandboxRuntime({ sourceType: runningAppInfo.sourceType, sourceId: runningAppInfo.sourceId, userId: uid, chatId, teamId: runningAppInfo.teamId, - tmbId: runningAppInfo.tmbId, + tmbId: runningUserInfo.tmbId, needSandboxRuntime: effectiveUseAgentSandbox, sandboxEntrypoint: effectiveSandboxEntrypoint, skillIds: effectiveSkillIds, + dynamicSkills, selectedSkills: effectiveSelectedSkills, editSkillId, prepareActions: agentSandboxPrepareActions, @@ -242,8 +252,9 @@ export const dispatchRunAgent = async (props: DispatchAgentModuleProps): Promise promptToolReferenceInfoMap } = await getSubapps({ tools: selectedTools, - tmbId: runningAppInfo.tmbId, - lang + tmbId: runningUserInfo.tmbId, + lang, + dynamic: dynamicTools }); const { getSubAppInfo, getSubApp } = createAgentSubAppLookup({ subAppsMap: agentSubAppsMap, @@ -262,6 +273,7 @@ export const dispatchRunAgent = async (props: DispatchAgentModuleProps): Promise context: { ...props, modelData: agentModel, + dynamicDataset, systemPrompt: formattedUserSystemPrompt, getSubAppInfo, getSubApp, diff --git a/packages/service/core/workflow/dispatch/ai/agent/sub/app/index.ts b/packages/service/core/workflow/dispatch/ai/agent/sub/app/index.ts index cb44c6cae52c..97fa718bc71b 100644 --- a/packages/service/core/workflow/dispatch/ai/agent/sub/app/index.ts +++ b/packages/service/core/workflow/dispatch/ai/agent/sub/app/index.ts @@ -1,7 +1,9 @@ import type { DispatchSubAppResponse } from '../../type'; -import { authAppByTmbId } from '../../../../../../../support/permission/app/auth'; -import { ReadPermissionVal } from '@fastgpt/global/support/permission/constant'; import { getAppVersionById } from '../../../../../../../core/app/version/controller'; +import { + createWorkflowChildResourceContext, + loadWorkflowAppResource +} from '../../../../../../../core/workflow/utils/resource'; import { getUserChatInfo } from '../../../../../../../support/user/team/utils'; import { runWorkflow } from '../../../../../../../core/workflow/dispatch'; import { @@ -71,6 +73,8 @@ type Props = Pick< }; userChatInput: string; customAppVariables: Record; + dynamic?: boolean; + useResourceSnapshot?: boolean; }; export const dispatchApp = async (props: Props): Promise => { @@ -81,20 +85,26 @@ export const dispatchApp = async (props: Props): Promise variableState, customAppVariables, userChatInput, + dynamic = false, ...data } = props; - // Auth the app by tmbId(Not the user, but the workflow user) - const { app: appData } = await authAppByTmbId({ + const appData = await loadWorkflowAppResource({ appId: app.id, - tmbId: runningAppInfo.tmbId, - per: ReadPermissionVal + tmbId: runningUserInfo.tmbId, + type: 'tool', + dynamic }); - const { nodes, edges, chatConfig } = await getAppVersionById({ + const childVersion = await getAppVersionById({ appId: app.id, versionId: app.version, app: appData }); + const { nodes, edges, chatConfig } = childVersion; + const resourceContext = await createWorkflowChildResourceContext( + childVersion.resources, + String(appData.teamId) + ); const workflowToolVariables = filterWorkflowToolInputVariables({ inputs: appData2FlowNodeIO({ chatConfig }).inputs, variables: customAppVariables @@ -125,6 +135,7 @@ export const dispatchApp = async (props: Props): Promise variablesConfig: chatConfig.variables ?? [], inputVariables: workflowToolVariables }), + resourceContext, fn: async ({ resolveInputFile }) => { const childrenVariableState = await WorkflowVariableState.create({ timezone: data.timezone, @@ -214,11 +225,14 @@ export const dispatchPlugin = async (props: Props): Promise> | null = null; const { nodes, edges, chatConfig, childAppInfo, externalProviderTmbId, billingTool } = await (async () => { if (app.systemToolId) { @@ -243,17 +257,24 @@ export const dispatchPlugin = async (props: Props): Promise { const childrenVariableState = await WorkflowVariableState.create({ timezone: data.timezone, diff --git a/packages/service/core/workflow/dispatch/ai/agent/sub/dataset/index.ts b/packages/service/core/workflow/dispatch/ai/agent/sub/dataset/index.ts index a03f215fbeed..9f02cf46ffe7 100644 --- a/packages/service/core/workflow/dispatch/ai/agent/sub/dataset/index.ts +++ b/packages/service/core/workflow/dispatch/ai/agent/sub/dataset/index.ts @@ -13,7 +13,6 @@ import { calculateCompressionThresholds } from '../../../../../../ai/llm/compres import { formatModelChars2Points } from '../../../../../../../support/wallet/usage/utils'; import { i18nT } from '@fastgpt/global/common/i18n/utils'; import { DatasetSearchModeEnum } from '@fastgpt/global/core/dataset/constants'; -import { MongoDataset } from '../../../../../../dataset/schema'; import { defaultSearchDatasetData, type DefaultSearchDatasetDataProps @@ -32,6 +31,7 @@ import { createQueryExtensionChildNodeResponse } from '../../../../dataset/nodeResponse'; import { filterDatasetsByTmbId } from '../../../../../../dataset/utils'; +import { loadWorkflowDatasetResource } from '../../../../../utils/resource'; import { normalizeDatasetSearchInput } from '../../../../dataset/utils'; import type { LLMSystemModelDataType } from '@fastgpt/global/core/ai/model.schema'; const logger = getLogger(LogCategories.MODULE.AI.AGENT); @@ -43,6 +43,7 @@ type DatasetSearchParams = { llmModel: LLMSystemModelDataType; userKey?: OpenaiAccountType; datasetParams?: AppFormEditFormType['dataset']; + dynamicDataset?: boolean; }; /** @@ -175,7 +176,8 @@ export const dispatchAgentDatasetSearch = async ({ teamId, tmbId, llmModel, - userKey + userKey, + dynamicDataset = false }: DatasetSearchParams): Promise => { if (!datasetParams || datasetParams.datasets.length === 0) { return { @@ -205,12 +207,14 @@ export const dispatchAgentDatasetSearch = async ({ }); try { - const datasetIds = datasetParams.authTmbId - ? await filterDatasetsByTmbId({ - datasetIds: datasetParams.datasets.map((item) => item.datasetId), - tmbId - }) - : datasetParams.datasets.map((item) => item.datasetId); + const requestedDatasetIds = datasetParams.datasets.map((item) => item.datasetId); + const datasetIds = + datasetParams.authTmbId || dynamicDataset + ? await filterDatasetsByTmbId({ + datasetIds: requestedDatasetIds, + tmbId + }) + : requestedDatasetIds; if (datasetIds.length === 0) { return { @@ -219,10 +223,12 @@ export const dispatchAgentDatasetSearch = async ({ } // Get vector model - const dataset = await MongoDataset.findById( - datasetIds[0], - 'vectorModelId vectorModel vlmModelId vlmModel' - ).lean(); + const dataset = await loadWorkflowDatasetResource({ + datasetId: datasetIds[0], + datasetIds: requestedDatasetIds, + dynamic: dynamicDataset, + tmbId + }); const vectorModel = getEmbeddingModelData({ modelId: dataset?.vectorModelId, model: dataset?.vectorModel diff --git a/packages/service/core/workflow/dispatch/ai/agent/sub/sandbox/prepare.ts b/packages/service/core/workflow/dispatch/ai/agent/sub/sandbox/prepare.ts index 712f83f361e5..befbf506a3c1 100644 --- a/packages/service/core/workflow/dispatch/ai/agent/sub/sandbox/prepare.ts +++ b/packages/service/core/workflow/dispatch/ai/agent/sub/sandbox/prepare.ts @@ -42,6 +42,7 @@ type EnsureAgentSandboxRuntimeParams = { needSandboxRuntime: boolean; sandboxEntrypoint?: string; skillIds: string[]; + dynamicSkills?: boolean; selectedSkills?: SelectedAgentSkillItemType[]; editSkillId?: string; prepareActions?: AgentSandboxPrepareAction[]; @@ -69,6 +70,7 @@ export async function ensureAgentSandboxRuntime({ needSandboxRuntime, sandboxEntrypoint, skillIds, + dynamicSkills = false, selectedSkills, editSkillId, prepareActions = [], @@ -109,7 +111,13 @@ export async function ensureAgentSandboxRuntime({ : prepareSandbox( context, preparePackageMirrors(), - injectSelectedSkillFiles({ teamId, tmbId, skillIds, selectedSkills }), + injectSelectedSkillFiles({ + teamId, + tmbId, + skillIds, + selectedSkills, + dynamicSkills + }), injectCurrentInputFiles(currentFiles, readInputFile), ...prepareActions, runSandboxEntrypoint({ sandboxEntrypoint }), @@ -179,12 +187,14 @@ const injectSelectedSkillFiles = teamId, tmbId, skillIds, - selectedSkills + selectedSkills, + dynamicSkills }: { teamId: string; tmbId: string; skillIds: string[]; selectedSkills?: SelectedAgentSkillItemType[]; + dynamicSkills: boolean; }): AgentSandboxPrepareStep => async (context) => { const deployedSkillVersions = await injectAgentSkillFilesToSandbox({ @@ -192,7 +202,8 @@ const injectSelectedSkillFiles = teamId, tmbId, skillIds, - workDirectory: context.workspaceRoot ?? context.workDirectory + workDirectory: context.workspaceRoot ?? context.workDirectory, + dynamic: dynamicSkills }); const selectedSkillMap = new Map(selectedSkills?.map((skill) => [skill.skillId, skill]) || []); diff --git a/packages/service/core/workflow/dispatch/ai/agent/sub/tool/index.ts b/packages/service/core/workflow/dispatch/ai/agent/sub/tool/index.ts index 2f1f5f66c890..b43f29ccef2e 100644 --- a/packages/service/core/workflow/dispatch/ai/agent/sub/tool/index.ts +++ b/packages/service/core/workflow/dispatch/ai/agent/sub/tool/index.ts @@ -19,10 +19,7 @@ import { assertToolRuntimeParams } from '@fastgpt/global/core/app/tool/runtime'; import { getHTTPToolRuntimeSchemas } from '@fastgpt/global/core/app/tool/httpTool/utils'; import { assertMCPUrlNotInternal, getMCPChildren, MCPClient } from '../../../../../../app/mcp'; import { getHTTPToolList, runHTTPTool } from '../../../../../../app/http'; -import { - decodeHttpToolSetNodesFromStorage, - decodeMcpToolSetNodesFromStorage -} from '../../../../../../app/jsonSchemaStorage'; +import { getAppVersionById } from '../../../../../../app/version/controller'; import { HttpToolSetRuntimeConfigSchema, McpToolSetRuntimeConfigSchema @@ -35,8 +32,12 @@ import { SystemToolRepo } from '../../../../../../app/tool/systemTool/systemTool import { computedSystemToolUsage } from '../../../../../../app/tool/runtime/utils'; import { InvokeProcessor } from '../../../../../../../support/invoke/invoke'; import { getLogger, LogCategories } from '../../../../../../../common/logger'; -import { authAppByTmbId } from '../../../../../../../support/permission/app/auth'; -import { ReadPermissionVal } from '@fastgpt/global/support/permission/constant'; +import { + filterWorkflowToolList, + isWorkflowResourceError, + loadWorkflowAppResource +} from '../../../../../utils/resource'; +import { getWorkflowResourceContext } from '../../../../../utils/context'; import { getWorkflowAppId } from '../../../../utils/source'; import { assertTeamPluginSourceAccess, @@ -64,6 +65,7 @@ export type Props = { uid: ChatDispatchProps['uid']; variableState: ChatDispatchProps['variableState']; workflowStreamResponse: ChatDispatchProps['workflowStreamResponse']; + dynamic?: boolean; }; export const dispatchTool = async ({ @@ -74,8 +76,10 @@ export const dispatchTool = async ({ chatId, uid, variableState, - workflowStreamResponse + workflowStreamResponse, + dynamic = false }: Props): Promise => { + const resourceContext = dynamic ? undefined : getWorkflowResourceContext(); const getNodeResponse = ({ result, response @@ -134,14 +138,14 @@ export const dispatchTool = async ({ * Agent 工具调用也会按持久化 toolId 解析 HTTP/MCP 父工具集。 * 这里必须使用当前运行工作流的 tmbId 做运行时授权,防止绕过保存阶段的脏引用被模型调用执行。 */ - const authRuntimeToolset = async (parentId: string) => - ( - await authAppByTmbId({ - tmbId: runningAppInfo.tmbId, - appId: parentId, - per: ReadPermissionVal - }) - ).app; + const authRuntimeToolset = async (parentId: string, toolName?: string) => + loadWorkflowAppResource({ + tmbId: runningUserInfo.tmbId, + appId: parentId, + type: 'tool', + toolName, + dynamic + }); if (toolConfig?.systemTool?.toolId) { const toolSource = getSystemToolSource(); @@ -256,12 +260,17 @@ export const dispatchTool = async ({ if (!parentId || !toolName) { return Promise.reject(`Invalid MCP tool id: ${toolConfig.mcpTool.toolId}`); } - const app = await authRuntimeToolset(parentId); + const app = await authRuntimeToolset(parentId, toolName); + const workflow = await getAppVersionById({ appId: parentId, app }); const mcpToolSet = McpToolSetRuntimeConfigSchema.safeParse( - decodeMcpToolSetNodesFromStorage(app.modules)[0]?.toolConfig?.mcpToolSet + workflow.nodes[0]?.toolConfig?.mcpToolSet ).data; - const mcpToolList = await getMCPChildren(app); - if (!mcpToolSet && !mcpToolList.length) { + const mcpToolList = filterWorkflowToolList({ + context: resourceContext, + appId: parentId, + tools: await getMCPChildren(app, workflow) + }); + if (!mcpToolSet || !mcpToolList.length) { return Promise.reject(`MCP tool set is missing`); } const mcpTool = getToolNameCandidates(toolName) @@ -302,12 +311,21 @@ export const dispatchTool = async ({ if (!parentId || !toolName) { return Promise.reject(`Invalid HTTP tool id: ${toolConfig.httpTool.toolId}`); } - const app = await authRuntimeToolset(parentId); + const app = await authRuntimeToolset(parentId, toolName); + const workflow = await getAppVersionById({ + appId: parentId, + versionId: version, + app + }); const toolSetData = HttpToolSetRuntimeConfigSchema.safeParse( - decodeHttpToolSetNodesFromStorage(app.modules)[0]?.toolConfig?.httpToolSet + workflow.nodes[0]?.toolConfig?.httpToolSet ).data; - const toolList = await getHTTPToolList(app); - if (!toolSetData && !toolList.length) { + const toolList = filterWorkflowToolList({ + context: resourceContext, + appId: parentId, + tools: await getHTTPToolList(app, workflow) + }); + if (!toolSetData || !toolList.length) { return Promise.reject(`HTTP tool set not found: ${toolConfig.httpTool.toolId}`); } @@ -360,6 +378,7 @@ export const dispatchTool = async ({ return getErrResponse("Can't find the tool"); } } catch (error) { + if (isWorkflowResourceError(error)) throw error; if (toolConfig?.systemTool?.toolId) { pushTrack.runSystemTool({ teamId: runningUserInfo.teamId, diff --git a/packages/service/core/workflow/dispatch/ai/agent/sub/tool/utils.ts b/packages/service/core/workflow/dispatch/ai/agent/sub/tool/utils.ts index d5016ba7e674..dcea64112537 100644 --- a/packages/service/core/workflow/dispatch/ai/agent/sub/tool/utils.ts +++ b/packages/service/core/workflow/dispatch/ai/agent/sub/tool/utils.ts @@ -10,8 +10,6 @@ import { splitToolsetToolPluginId } from '@fastgpt/global/core/app/tool/utils'; import type { localeType } from '@fastgpt/global/common/i18n/type'; -import { ReadPermissionVal } from '@fastgpt/global/support/permission/constant'; -import { authAppByTmbId } from '../../../../../../../support/permission/app/auth'; import { getErrText } from '@fastgpt/global/common/error/utils'; import { FlowNodeInputTypeEnum, @@ -51,6 +49,7 @@ import { toolData2FlowNodeIO } from '@fastgpt/global/core/workflow/utils'; import type { AppSchemaType } from '@fastgpt/global/core/app/type'; +import type { AppVersionSchemaType } from '@fastgpt/global/core/app/version/type'; import { getAppVersionById } from '../../../../../../app/version/controller'; import { AppFolderTypeList, AppTypeEnum } from '@fastgpt/global/core/app/constants'; import { PluginErrEnum } from '@fastgpt/global/common/error/code/plugin'; @@ -58,9 +57,15 @@ import { parseI18nString } from '@fastgpt/global/common/i18n/utils'; import { SystemToolRepo } from '../../../../../../app/tool/systemTool/systemTool.repo'; import { Output_Template_Error_Message } from '@fastgpt/global/core/workflow/template/output'; import type { NodeToolConfigType } from '@fastgpt/global/core/workflow/type/node'; +import { HttpToolSetRuntimeConfigSchema } from '@fastgpt/global/core/workflow/type/node'; import { getMCPChildren } from '../../../../../../app/mcp'; -import { getHTTPToolList } from '../../../../../../app/http'; -import { decodeToolSetNodesFromStorage } from '../../../../../../app/jsonSchemaStorage'; +import { + filterWorkflowToolList, + getWorkflowAppWorkflow, + isWorkflowResourceError, + loadWorkflowAppResource +} from '../../../../../utils/resource'; +import { getWorkflowResourceContext } from '../../../../../utils/context'; import { getTmbInfoByTmbId } from '../../../../../../../support/user/team/controller'; import { assertTeamPluginSourceAccess, @@ -94,12 +99,15 @@ const getAgentRuntimeToolId = ({ pluginId, source }: { pluginId: string; source? export const getAgentRuntimeTools = async ({ tools, tmbId, - lang + lang, + dynamic = false }: { tools: AgentToolType[]; tmbId: string; lang?: localeType; + dynamic?: boolean; }): Promise => { + const resourceContext = dynamic ? undefined : getWorkflowResourceContext(); let teamIdPromise: Promise | undefined; const getCurrentTeamId = async () => { if (!teamIdPromise) { @@ -121,7 +129,7 @@ export const getAgentRuntimeTools = async ({ const getToolSetNodeIO = ({ nodes }: { - nodes: AppSchemaType['modules']; + nodes: AppVersionSchemaType['nodes']; }): { inputs: RuntimeNodeItemType['inputs']; outputs: RuntimeNodeItemType['outputs']; @@ -249,7 +257,7 @@ export const getAgentRuntimeTools = async ({ }; }; - // 缺失或空版本都表示最新版本;固定版本由 Agent 工具配置显式传入。 + // 所有 App,包括 MCP/HTTP 工具集,都从对应 Version 读取;App.modules 只属于迁移数据。 const getVersionNodes = async ({ app, versionId @@ -257,16 +265,6 @@ export const getAgentRuntimeTools = async ({ app: AppSchemaType; versionId?: string; }) => { - if (app.type === AppTypeEnum.mcpToolSet || app.type === AppTypeEnum.httpToolSet) { - return { - versionId: undefined, - versionName: undefined, - nodes: decodeToolSetNodesFromStorage(app.modules), - edges: app.edges, - chatConfig: app.chatConfig - }; - } - const version = await getAppVersionById({ appId: String(app._id), versionId, @@ -366,8 +364,7 @@ export const getAgentRuntimeTools = async ({ /** * 解析单个 MCP 工具 id: mcp-${appId}/${toolName}。 - * 运行时始终从 MCP 工具集应用读取完整工具定义;旧版子 App 数据由 getMCPChildren - * 内部兼容处理。 + * 运行时始终从当前 MCP 工具集 Version 读取完整工具定义。 */ const formatMcpToolNode = async ({ app, @@ -377,7 +374,11 @@ export const getAgentRuntimeTools = async ({ pluginId: string; }): Promise => { const { toolName } = splitToolsetToolPluginId(pluginId); - const toolList = await getMCPChildren(app); + const toolList = filterWorkflowToolList({ + context: resourceContext, + appId: String(app._id), + tools: await getMCPChildren(app, getWorkflowAppWorkflow(String(app._id))) + }); const tool = findToolByName(toolList, toolName); if (!tool) return Promise.reject(PluginErrEnum.unExist); @@ -410,7 +411,12 @@ export const getAgentRuntimeTools = async ({ pluginId: string; }): Promise => { const { toolName } = splitToolsetToolPluginId(pluginId); - const toolList = await getHTTPToolList(app); + const version = await getVersionNodes({ app, versionId: undefined }); + const toolList = filterWorkflowToolList({ + context: resourceContext, + appId: String(app._id), + tools: version.nodes[0]?.toolConfig?.httpToolSet?.toolList ?? [] + }); const tool = getToolNameCandidates(toolName) .map((name) => toolList.find((item) => item.name === name)) .find(Boolean); @@ -509,13 +515,12 @@ export const getAgentRuntimeTools = async ({ }) => { if (!app || !toolSetId || String(app._id) === toolSetId) return app; - return ( - await authAppByTmbId({ - tmbId, - appId: toolSetId, - per: ReadPermissionVal - }) - ).app; + return loadWorkflowAppResource({ + tmbId, + appId: toolSetId, + type: 'tool', + dynamic + }); }; return Promise.all( @@ -528,11 +533,12 @@ export const getAgentRuntimeTools = async ({ tool.toolConfig?.systemToolSet?.source; // 工具间整体并发;单个 App 类工具必须先鉴权拿到 app,才能读取对应版本节点。 const authAppPromise = authAppId - ? authAppByTmbId({ + ? loadWorkflowAppResource({ tmbId, appId: authAppId, - per: ReadPermissionVal - }) + type: idSource === AppToolSourceEnum.personal ? 'agent' : 'tool', + dynamic + }).then((app) => ({ app })) : Promise.resolve(undefined); const [authResult, toolNode] = await Promise.all([ @@ -698,6 +704,7 @@ export const getAgentRuntimeTools = async ({ return { type: 'tool', id: runtimeId, + dynamic, name: child.name, avatar: child.avatar, // MCP/HTTP tools have no workflow version. @@ -743,7 +750,16 @@ export const getAgentRuntimeTools = async ({ app: authApp, toolSetId }); - const finalToolList = toolSetApp ? await getMCPChildren(toolSetApp) : []; + const finalToolList = toolSetApp + ? filterWorkflowToolList({ + context: resourceContext, + appId: String(toolSetApp._id), + tools: await getMCPChildren( + toolSetApp, + getWorkflowAppWorkflow(String(toolSetApp._id)) + ) + }) + : []; const runtimeToolSetId = toolSetId || toolNode.pluginId || pluginId; const children = finalToolList.map((tool, index) => { @@ -766,7 +782,17 @@ export const getAgentRuntimeTools = async ({ app: authApp, toolSetId }); - const toolList = toolSetApp ? await getHTTPToolList(toolSetApp) : []; + const toolList = toolSetApp + ? filterWorkflowToolList({ + context: resourceContext, + appId: String(toolSetApp._id), + tools: + HttpToolSetRuntimeConfigSchema.safeParse( + (await getVersionNodes({ app: toolSetApp, versionId: undefined })).nodes[0] + ?.toolConfig?.httpToolSet + ).data?.toolList ?? [] + }) + : []; const runtimeToolSetId = toolSetId || toolNode.pluginId || pluginId; const children = toolList.map((tool: HttpToolConfigType, index) => { const newToolNode = getHTTPToolRuntimeNode({ @@ -803,6 +829,7 @@ export const getAgentRuntimeTools = async ({ { type: toolType, id: runtimeToolId, + dynamic, name: toolNode.name, avatar: toolNode.avatar, version: toolNode.version, @@ -816,6 +843,7 @@ export const getAgentRuntimeTools = async ({ ]; } } catch (error) { + if (isWorkflowResourceError(error)) throw error; getLogger(LogCategories.MODULE.AI.AGENT).warn(`[Agent] tool load error`, { toolId: tool.id, error: getErrText(error) diff --git a/packages/service/core/workflow/dispatch/ai/agent/sub/type.ts b/packages/service/core/workflow/dispatch/ai/agent/sub/type.ts index 25ae52caea9c..efb807223417 100644 --- a/packages/service/core/workflow/dispatch/ai/agent/sub/type.ts +++ b/packages/service/core/workflow/dispatch/ai/agent/sub/type.ts @@ -11,6 +11,7 @@ export type SubAppInitType = { name: string; avatar?: string; version?: string; + dynamic?: boolean; toolConfig?: RuntimeNodeItemType['toolConfig']; inputs: RuntimeNodeItemType['inputs']; agentGeneratedInputKeys: string[]; diff --git a/packages/service/core/workflow/dispatch/ai/agent/sub/utils.ts b/packages/service/core/workflow/dispatch/ai/agent/sub/utils.ts index 5790270a6a2d..d80c3630486b 100644 --- a/packages/service/core/workflow/dispatch/ai/agent/sub/utils.ts +++ b/packages/service/core/workflow/dispatch/ai/agent/sub/utils.ts @@ -24,11 +24,13 @@ import { mergeToolRuntimeParams } from '@fastgpt/global/core/app/tool/runtime'; export const getSubapps = async ({ tmbId, tools, - lang + lang, + dynamic = false }: { tmbId: string; tools: AgentToolType[]; lang?: localeType; + dynamic?: boolean; }): Promise<{ completionTools: ChatCompletionTool[]; subAppsMap: Map; @@ -42,7 +44,8 @@ export const getSubapps = async ({ const formatTools = await getAgentRuntimeTools({ tools, tmbId, - lang + lang, + dynamic }); formatTools.forEach((tool) => { if (tool.promptReference) { @@ -60,6 +63,7 @@ export const getSubapps = async ({ name: tool.name, avatar: tool.avatar, version: tool.version, + dynamic: tool.dynamic, toolConfig: tool.toolConfig, inputs: tool.inputs, agentGeneratedInputKeys: tool.agentGeneratedInputKeys, @@ -93,6 +97,7 @@ export type ToolDispatchContext = Pick< | 'retainDatasetCite' | 'maxRunTimes' | 'workflowDispatchDeep' + | 'dynamicDataset' | 'params' | 'stream' | 'nodeResponseSink' @@ -183,6 +188,7 @@ export const getExecuteTool = ({ const { response, + assistantMessages, usages = [], interactive, stop = false, @@ -190,6 +196,7 @@ export const getExecuteTool = ({ errorMessage } = await (async (): Promise<{ response: string; + assistantMessages?: DispatchSubAppResponse['assistantMessages']; usages?: ChatNodeUsageType[]; interactive?: DispatchSubAppResponse['interactive']; stop?: boolean; @@ -231,6 +238,7 @@ export const getExecuteTool = ({ version: tool.version, toolConfig: tool.toolConfig }, + dynamic: tool.dynamic, params: requestParams, runningUserInfo, runningAppInfo, @@ -250,37 +258,41 @@ export const getExecuteTool = ({ } else if (tool.type === 'workflow') { const { userChatInput, ...params } = filterAgentWorkflowRuntimeParams(requestParams); - const { response, usages, interactive, nodeResponse, errorMessage } = await dispatchApp({ - app: { - name: tool.name, - avatar: tool.avatar, - id: tool.id, - version: tool.version - }, - userChatInput: userChatInput, - customAppVariables: params, - checkIsStopping, - lang, - requestOrigin, - mode, - timezone, - externalProvider, - chatId, - responseChatItemId, - uid, - runningAppInfo, - runningUserInfo, - retainDatasetCite, - maxRunTimes, - workflowDispatchDeep, - nodeResponseSink, - nodeResponseParentId: callId, - variableState, - lastInteractive - }); + const { response, assistantMessages, usages, interactive, nodeResponse, errorMessage } = + await dispatchApp({ + app: { + name: tool.name, + avatar: tool.avatar, + id: tool.id, + version: tool.version + }, + dynamic: tool.dynamic, + userChatInput: userChatInput, + customAppVariables: params, + checkIsStopping, + lang, + requestOrigin, + mode, + timezone, + externalProvider, + chatId, + responseChatItemId, + uid, + runningAppInfo, + runningUserInfo, + retainDatasetCite, + maxRunTimes, + workflowDispatchDeep, + nodeResponseSink, + nodeResponseParentId: callId, + variableState, + lastInteractive, + useResourceSnapshot: true + }); return { response, + assistantMessages, usages, interactive, nodeResponse, @@ -313,7 +325,7 @@ export const getExecuteTool = ({ }; })(); const customAppVariables = filterAgentWorkflowRuntimeParams(requestParams); - const { response, usages, interactive, nodeResponse, errorMessage } = + const { response, assistantMessages, usages, interactive, nodeResponse, errorMessage } = await dispatchPlugin({ app: { name: tool.name, @@ -322,6 +334,7 @@ export const getExecuteTool = ({ version: tool.version, ...(systemToolId ? { systemToolId } : {}) }, + dynamic: tool.dynamic, userChatInput: '', customAppVariables, checkIsStopping, @@ -341,11 +354,13 @@ export const getExecuteTool = ({ nodeResponseSink, nodeResponseParentId: callId, variableState, - lastInteractive + lastInteractive, + useResourceSnapshot: tool.type !== 'commercialTool' }); return { response, + assistantMessages, usages, interactive, nodeResponse, @@ -391,6 +406,7 @@ export const getExecuteTool = ({ return { response, + assistantMessages, usages, interactive, stop, diff --git a/packages/service/core/workflow/dispatch/ai/agent/toolProvider/createWorkflowAgentToolProvider.ts b/packages/service/core/workflow/dispatch/ai/agent/toolProvider/createWorkflowAgentToolProvider.ts index e7594ddddbb4..6292967428fe 100644 --- a/packages/service/core/workflow/dispatch/ai/agent/toolProvider/createWorkflowAgentToolProvider.ts +++ b/packages/service/core/workflow/dispatch/ai/agent/toolProvider/createWorkflowAgentToolProvider.ts @@ -39,7 +39,8 @@ export const createWorkflowAgentToolProvider = ({ teamId: context.runningUserInfo.teamId, tmbId: context.runningUserInfo.tmbId, llmModel: context.modelData, - userKey: context.externalProvider.openaiAccount + userKey: context.externalProvider.openaiAccount, + dynamicDataset: context.dynamicDataset }); const usages = result.usages ?? []; const datasetSearchInfo = context.getSubAppInfo(SubAppIds.datasetSearch); diff --git a/packages/service/core/workflow/dispatch/ai/agent/type.ts b/packages/service/core/workflow/dispatch/ai/agent/type.ts index 276f5bd59e34..d3d127a7e99d 100644 --- a/packages/service/core/workflow/dispatch/ai/agent/type.ts +++ b/packages/service/core/workflow/dispatch/ai/agent/type.ts @@ -25,6 +25,7 @@ export const SubAppRuntimeSchema = z.object({ avatar: z.string().optional(), toolDescription: z.string().optional(), version: z.string().optional(), + dynamic: z.boolean().optional(), toolConfig: NodeToolConfigTypeSchema.optional(), inputs: z.custom().optional(), agentGeneratedInputKeys: z.array(z.string()).optional(), diff --git a/packages/service/core/workflow/dispatch/child/runApp.ts b/packages/service/core/workflow/dispatch/child/runApp.ts index 63ac3cf98d88..2aa562f5eac2 100644 --- a/packages/service/core/workflow/dispatch/child/runApp.ts +++ b/packages/service/core/workflow/dispatch/child/runApp.ts @@ -16,10 +16,9 @@ import { getNodeErrResponse, getHistories } from '../utils'; import { getWorkflowFileVariableInputs, WorkflowVariableState } from '../utils/variables'; import { chatValue2RuntimePrompt, runtimePrompt2ChatsValue } from '@fastgpt/global/core/chat/adapt'; import type { DispatchNodeResultType, ModuleDispatchProps } from '../../types/runtime'; -import { authAppByTmbId } from '../../../../support/permission/app/auth'; -import { ReadPermissionVal } from '@fastgpt/global/support/permission/constant'; import { getAppVersionById } from '../../../app/version/controller'; import { parseUrlToFileType, runWithDerivedWorkflowFileContext } from '../../utils/context'; +import { createWorkflowChildResourceContext, loadWorkflowAppResource } from '../../utils/resource'; import { getUserChatInfo } from '../../../../support/user/team/utils'; import { getRunningUserInfoByTmbId } from '../../../../support/user/team/utils'; import { getRuntimeNodeResponseSummary } from '../utils'; @@ -75,17 +74,21 @@ export const dispatchRunAppNode = async (props: Props): Promise => { } try { - // Auth the app by tmbId(Not the user, but the workflow user) - const { app: appData } = await authAppByTmbId({ - appId: appId, - tmbId: runningAppInfo.tmbId, - per: ReadPermissionVal + const appData = await loadWorkflowAppResource({ + appId, + tmbId: props.runningUserInfo.tmbId, + type: 'agent' }); - const { nodes, edges, chatConfig } = await getAppVersionById({ + const childVersion = await getAppVersionById({ appId, versionId: version, app: appData }); + const { nodes, edges, chatConfig } = childVersion; + const resourceContext = await createWorkflowChildResourceContext( + childVersion.resources, + String(appData.teamId) + ); const childStreamResponse = system_forbid_stream ? false : props.stream; // Auto line @@ -140,6 +143,7 @@ export const dispatchRunAppNode = async (props: Props): Promise => { variablesConfig: chatConfig.variables, inputVariables: childrenAppVariables }), + resourceContext, fn: async ({ resolveInputFile, query: childQuery, histories: childHistories }) => { filteredChildHistories = childHistories; filteredChildQuery = childQuery; diff --git a/packages/service/core/workflow/dispatch/child/runTool.ts b/packages/service/core/workflow/dispatch/child/runTool.ts index c0bb7b4c19bb..61f618100c85 100644 --- a/packages/service/core/workflow/dispatch/child/runTool.ts +++ b/packages/service/core/workflow/dispatch/child/runTool.ts @@ -15,15 +15,18 @@ import type { StoreSecretValueType } from '@fastgpt/global/common/secret/type'; import { pushTrack } from '../../../../common/middle/tracks/utils'; import { getNodeErrResponse } from '../utils'; import { getHTTPToolList, runHTTPTool } from '../../../app/http'; -import { - decodeHttpToolSetNodesFromStorage, - decodeMcpToolSetNodesFromStorage -} from '../../../app/jsonSchemaStorage'; +import { getWorkflowContext } from '../../utils/context'; +import { getWorkflowResourceContext } from '../../utils/context'; +import { getAppVersionById } from '../../../app/version/controller'; import { HttpToolSetRuntimeConfigSchema, McpToolSetRuntimeConfigSchema } from '@fastgpt/global/core/workflow/type/node'; -import { getWorkflowContext } from '../../utils/context'; +import { + filterWorkflowToolList, + isWorkflowResourceError, + loadWorkflowAppResource +} from '../../utils/resource'; import { getToolNameCandidates, getToolRawId, @@ -35,8 +38,6 @@ import { SystemToolRepo } from '../../../app/tool/systemTool/systemTool.repo'; import { computedSystemToolUsage } from '../../../app/tool/runtime/utils'; import { InvokeProcessor } from '../../../../support/invoke/invoke'; import { getLogger, LogCategories } from '../../../../common/logger'; -import { authAppByTmbId } from '../../../../support/permission/app/auth'; -import { ReadPermissionVal } from '@fastgpt/global/support/permission/constant'; import { getWorkflowAppId } from '../utils/source'; import { assertTeamPluginSourceAccess, @@ -76,6 +77,7 @@ export const dispatchRunTool = async (props: RunToolProps): Promise { - return ( - await authAppByTmbId({ - tmbId: runningAppInfo.tmbId, - appId: parentId, - per: ReadPermissionVal - }) - ).app; - }; + const authRuntimeToolset = async (parentId: string, toolName?: string) => + loadWorkflowAppResource({ + tmbId: runningUserInfo.tmbId, + appId: parentId, + type: 'tool', + toolName + }); // run system tool if (toolConfig?.systemTool?.toolId) { @@ -283,12 +283,17 @@ export const dispatchRunTool = async (props: RunToolProps): Promise mcpToolList.find((tool) => tool.name === name)) .find(Boolean); @@ -328,12 +333,21 @@ export const dispatchRunTool = async (props: RunToolProps): Promise item.datasetId), - tmbId - }) - : await Promise.resolve(datasets.map((item) => item.datasetId)); + const dynamicDataset = nodeHasDynamicInput(node, [ + NodeInputKeyEnum.datasetSelectList, + NodeInputKeyEnum.datasetParams + ]); + const requestedDatasetIds = datasets.map((item) => item.datasetId); + const datasetIds = + authTmbId || dynamicDataset + ? await filterDatasetsByTmbId({ + datasetIds: requestedDatasetIds, + tmbId + }) + : requestedDatasetIds; if (datasetIds.length === 0) { return emptyResult; } // Get vector model - const dataset = await MongoDataset.findById( - datasets[0].datasetId, - 'vectorModelId vectorModel vlmModelId vlmModel' - ).lean(); + const dataset = await loadWorkflowDatasetResource({ + datasetId: datasetIds[0], + datasetIds: requestedDatasetIds, + dynamic: dynamicDataset, + tmbId + }); const vectorModel = getEmbeddingModelData({ modelId: dataset?.vectorModelId, model: dataset?.vectorModel diff --git a/packages/service/core/workflow/dispatch/index.ts b/packages/service/core/workflow/dispatch/index.ts index 7087c4998a1d..bebf8f8659e4 100644 --- a/packages/service/core/workflow/dispatch/index.ts +++ b/packages/service/core/workflow/dispatch/index.ts @@ -77,6 +77,7 @@ import { getWorkflowNodeRunParams } from './utils/runtime'; import type { AgentSandboxPrepareAction } from './ai/agent/sub/sandbox'; import { getWorkflowSource } from './utils/source'; import { prepareWorkflowFileContext } from '../utils/fileContext'; +import type { WorkflowResourceContext } from '../utils/resource'; const logger = getLogger(LogCategories.MODULE.WORKFLOW.DISPATCH); @@ -101,6 +102,8 @@ type Props = Omit< maxFileAmount: number; /** 已按团队配置优先、系统配置兜底计算完成的单文件读取上限。 */ maxBytesPerFile: number; + /** App 入口选中 Version 的资源快照;Test/Debug 传入当前请求临时快照。Skill 调试和商业工具可以不传。 */ + resourceContext?: WorkflowResourceContext; }; type NodeResponseType = DispatchNodeResultType<{ [key: string]: any; @@ -340,7 +343,8 @@ export async function dispatchWorkFlow({ { mcpClientMemory: {}, fileContext, - fileRegistrar + fileRegistrar, + resourceContext: data.resourceContext }, (ctx) => { runWorkflow({ @@ -1629,7 +1633,7 @@ export const runWorkflow = async (data: RunWorkflowProps): Promise { const childRuntimeNodes = runtimeNodes.map((node) => node.flowNodeType === FlowNodeTypeEnum.pluginInput diff --git a/packages/service/core/workflow/dispatch/utils/index.ts b/packages/service/core/workflow/dispatch/utils/index.ts index 8d7e7b23697c..60b44d91a014 100644 --- a/packages/service/core/workflow/dispatch/utils/index.ts +++ b/packages/service/core/workflow/dispatch/utils/index.ts @@ -33,6 +33,8 @@ import type { WorkflowResponseType } from '../type'; import { getLogger, LogCategories } from '../../../../common/logger'; import { parsetMcpToolConfig } from '@fastgpt/global/core/app/tool/mcpTool/utils'; import { getHTTPToolList } from '../../../app/http'; +import { getWorkflowResourceContext } from '../../utils/context'; +import { assertWorkflowResource, filterWorkflowToolList } from '../../utils/resource'; import { getLastInteractiveValue } from '@fastgpt/global/core/workflow/runtime/utils'; import { getSavedToolInputSelectedType, @@ -586,6 +588,12 @@ export const rewriteRuntimeWorkFlow = async ({ }; const getAuthorizedToolSet = async (toolSetId: string) => { + const resourceContext = getWorkflowResourceContext(); + if (resourceContext) { + assertWorkflowResource({ context: resourceContext, type: 'tool', id: toolSetId }); + return resourceContext.appMap.get(toolSetId); + } + try { return ( await authAppByTmbId({ @@ -654,7 +662,11 @@ export const rewriteRuntimeWorkFlow = async ({ const app = await getAuthorizedToolSet(toolSetId); if (!app) continue; - const toolList = (await getMCPChildren(app)) as RuntimeMcpTool[]; + const toolList = filterWorkflowToolList({ + context: getWorkflowResourceContext(), + appId: String(app._id), + tools: (await getMCPChildren(app)) as RuntimeMcpTool[] + }); toolList.forEach((tool, index) => { const newToolNode = initToolSetChildNode( @@ -678,7 +690,11 @@ export const rewriteRuntimeWorkFlow = async ({ const app = await getAuthorizedToolSet(toolSetId); if (!app) continue; - const toolList = await getHTTPToolList(app); + const toolList = filterWorkflowToolList({ + context: getWorkflowResourceContext(), + appId: String(app._id), + tools: await getHTTPToolList(app) + }); toolList.forEach((tool: HttpToolConfigType, index: number) => { const newToolNode = initToolSetChildNode( @@ -733,7 +749,11 @@ export const rewriteRuntimeWorkFlow = async ({ const toolListMap = new Map(); await Promise.all( toolsets.map(async (toolset) => { - const toolList = (await getMCPChildren(toolset)) as RuntimeMcpTool[]; + const toolList = filterWorkflowToolList({ + context: getWorkflowResourceContext(), + appId: String(toolset._id), + tools: (await getMCPChildren(toolset)) as RuntimeMcpTool[] + }); toolListMap.set(String(toolset._id), toolList); }) ); @@ -774,7 +794,14 @@ export const rewriteRuntimeWorkFlow = async ({ const toolListMap = new Map(); await Promise.all( toolsets.map(async (toolset) => { - toolListMap.set(String(toolset._id), await getHTTPToolList(toolset)); + toolListMap.set( + String(toolset._id), + filterWorkflowToolList({ + context: getWorkflowResourceContext(), + appId: String(toolset._id), + tools: await getHTTPToolList(toolset) + }) + ); }) ); httpToolNodes.forEach((node) => { diff --git a/packages/service/core/workflow/dispatch/utils/variables.ts b/packages/service/core/workflow/dispatch/utils/variables.ts index 84d8dd1c78f3..7759428e95fa 100644 --- a/packages/service/core/workflow/dispatch/utils/variables.ts +++ b/packages/service/core/workflow/dispatch/utils/variables.ts @@ -1,3 +1,4 @@ +import type { AppChatConfigType } from '@fastgpt/global/core/app/type'; import type { VariableItemType } from '@fastgpt/global/core/app/variable/type'; import type { ChatFileStoreValue } from '@fastgpt/global/core/chat/type'; import { VariableInputEnum } from '@fastgpt/global/core/workflow/constants'; @@ -62,7 +63,7 @@ export type WorkflowVariableStateCreateProps = { runtimeOnlyVariables?: Record; // 全局变量配置 - variablesConfig?: ChatDispatchProps['chatConfig']['variables']; + variablesConfig?: AppChatConfigType['variables']; // 外部传入的变量,需要基于 variablesConfig 加工 inputVariables?: Record; // 源变量状态,用于复制变量状态 diff --git a/packages/service/core/workflow/types/runtime.ts b/packages/service/core/workflow/types/runtime.ts index 74fa5d8a8cf0..80aca40064e2 100644 --- a/packages/service/core/workflow/types/runtime.ts +++ b/packages/service/core/workflow/types/runtime.ts @@ -1,5 +1,5 @@ import type { localeType } from '@fastgpt/global/common/i18n/type'; -import type { AppSchemaType } from '@fastgpt/global/core/app/type'; +import type { AppChatConfigType } from '@fastgpt/global/core/app/type'; import type { ReasoningEffort } from '@fastgpt/global/core/ai/llm/type'; import type { AiChatQuoteRoleType } from '@fastgpt/global/core/workflow/template/system/aiChat/type'; import type { @@ -90,7 +90,7 @@ export type ChatDispatchProps = { histories: ChatItemMiniType[]; variableState: WorkflowVariableStateLike; // global variable state query: UserChatItemValueItemType[]; // trigger query - chatConfig: AppSchemaType['chatConfig']; + chatConfig?: AppChatConfigType; lastInteractive?: WorkflowInteractiveResponseType; // last interactive response stream: boolean; retainDatasetCite?: boolean; @@ -102,6 +102,9 @@ export type ChatDispatchProps = { workflowDispatchDeep: number; + /** 当前节点的资源输入是否来自工作流引用,动态资源回退到运行人权限。 */ + dynamicDataset?: boolean; + responseAllData?: boolean; responseDetail?: boolean; nodeResponseParentId?: string; // 传递给 child,用于设置 nodeResponse 的 parentId diff --git a/packages/service/core/workflow/utils/context.ts b/packages/service/core/workflow/utils/context.ts index 4ffcdb5e787c..6350a0cef136 100644 --- a/packages/service/core/workflow/utils/context.ts +++ b/packages/service/core/workflow/utils/context.ts @@ -16,11 +16,13 @@ import type { ChatItemMiniType, UserChatItemValueItemType } from '@fastgpt/global/core/chat/type'; +import type { WorkflowResourceContext } from './resource'; type ContextType = { mcpClientMemory: Record; fileContext?: WorkflowFileContext; fileRegistrar?: WorkflowFileRegistrar; + resourceContext?: WorkflowResourceContext; }; export const WorkflowContext = new AsyncLocalStorage(); @@ -50,6 +52,9 @@ export const getWorkflowFileMaxAmount = () => { /** 获取只供 Workflow 输入适配器使用的文件登记能力。 */ export const getWorkflowFileRegistrar = () => WorkflowContext.getStore()?.fileRegistrar; +/** 获取当前 App 运行时的只读资源快照。非 App 来源返回 undefined。 */ +export const getWorkflowResourceContext = () => WorkflowContext.getStore()?.resourceContext; + /** * 为 Child 运行创建独立的聊天输入引用。 * @@ -94,11 +99,13 @@ export const runWithDerivedWorkflowFileContext = async ({ query = [], histories = [], files, + resourceContext, fn }: { query?: UserChatItemValueItemType[]; histories?: ChatItemMiniType[]; files: WorkflowFileInput[]; + resourceContext?: WorkflowResourceContext | null; fn: (scope: { resolveInputFile?: (file: ChatFileStoreValue) => Promise; query: UserChatItemValueItemType[]; @@ -110,10 +117,20 @@ export const runWithDerivedWorkflowFileContext = async ({ const parentFileContext = parentStore?.fileContext; if (!parentStore || !parentFileContext) { const childInputs = cloneChildChatInputs({ query, histories }); - return fn({ - ...childInputs, - filterFiles: (files) => cloneChildFiles(files) - }); + const runChild = () => + fn({ + ...childInputs, + filterFiles: (files) => cloneChildFiles(files) + }); + if (resourceContext === undefined) return runChild(); + + return runWithContext( + { + mcpClientMemory: parentStore?.mcpClientMemory ?? {}, + resourceContext: resourceContext ?? undefined + }, + runChild + ); } const maxFileAmount = Math.max(parentFileContext.limits.maxFileAmount, 0); @@ -191,7 +208,9 @@ export const runWithDerivedWorkflowFileContext = async ({ const childStore: ContextType = { ...parentStore, fileContext: childFileContext, - fileRegistrar: undefined + fileRegistrar: undefined, + resourceContext: + resourceContext === undefined ? parentStore.resourceContext : (resourceContext ?? undefined) }; const resolveInputFile = async (file: ChatFileStoreValue) => { diff --git a/packages/service/core/workflow/utils/fileLimits.ts b/packages/service/core/workflow/utils/fileLimits.ts index 97c9eb209298..a8037810ddbb 100644 --- a/packages/service/core/workflow/utils/fileLimits.ts +++ b/packages/service/core/workflow/utils/fileLimits.ts @@ -1,4 +1,4 @@ -import type { AppSchemaType } from '@fastgpt/global/core/app/type'; +import type { AppChatConfigType } from '@fastgpt/global/core/app/type'; import type { UserChatItemValueItemType } from '@fastgpt/global/core/chat/type'; import { getFileAmountLimit, @@ -86,7 +86,7 @@ export const prepareWorkflowFileQuery = async ({ limits }: { teamId: string; - chatConfig?: AppSchemaType['chatConfig']; + chatConfig?: AppChatConfigType; query: UserChatItemValueItemType[]; limits?: WorkflowFileLimits; }) => { diff --git a/packages/service/core/workflow/utils/resource.ts b/packages/service/core/workflow/utils/resource.ts new file mode 100644 index 000000000000..1d2b511e9646 --- /dev/null +++ b/packages/service/core/workflow/utils/resource.ts @@ -0,0 +1,297 @@ +import type { + AppResource, + AppResourcesType, + AppResourceType, + AppSchemaType +} from '@fastgpt/global/core/app/type'; +import type { DatasetSchemaType } from '@fastgpt/global/core/dataset/type'; +import type { AgentSkillSchemaType } from '@fastgpt/global/core/ai/skill/type'; +import { AgentSkillSourceEnum } from '@fastgpt/global/core/ai/skill/constants'; +import { AppErrEnum } from '@fastgpt/global/common/error/code/app'; +import { DatasetErrEnum } from '@fastgpt/global/common/error/code/dataset'; +import { UserError } from '@fastgpt/global/common/error/utils'; +import { ReadPermissionVal } from '@fastgpt/global/support/permission/constant'; +import { MongoApp } from '../../app/schema'; +import { MongoDataset } from '../../dataset/schema'; +import { MongoAgentSkills } from '../../ai/skill/model/schema'; +import { authAppByTmbId } from '../../../support/permission/app/auth'; +import { authDatasetByTmbId } from '../../../support/permission/dataset/auth'; +import { mergeAppResources } from '../../app/resources'; +import { getWorkflowResourceContext } from './context'; +import { + getAppPublishedWorkflowMap, + type AppPublishedWorkflow +} from '../../app/version/controller'; + +export type WorkflowResourceContext = { + teamId?: string; + /** root Test/Debug 请求允许子工作流沿用跨团队资源权限。 */ + isRoot: boolean; + resources: AppResourcesType; + resourceMap: Map; + appMap: Map; + workflowMap: Map; + datasetMap: Map; + skillMap: Map; +}; + +export type WorkflowResourceEntities = { + apps: AppSchemaType[]; + datasets: DatasetSchemaType[]; + skills: AgentSkillSchemaType[]; +}; + +/** 静态资源快照不一致错误;不能被工具加载器降级为单个工具不可用。 */ +export class WorkflowResourceError extends UserError {} + +export const isWorkflowResourceError = (error: unknown): error is WorkflowResourceError => + error instanceof WorkflowResourceError; + +const getResourceKey = (type: AppResourceType, id: string, modelType?: string) => + type === 'model' ? `${type}:${modelType}:${id}` : `${type}:${id}`; + +/** 按资源快照批量加载实体;root 调试请求跳过团队过滤,但仍校验实体存在。 */ +export const loadWorkflowResourceContext = async ({ + resources, + teamId, + isRoot = false +}: { + resources: AppResourcesType; + teamId?: string; + isRoot?: boolean; +}) => { + const normalizedResources = mergeAppResources(Array.isArray(resources) ? resources : []); + const resourceMap = new Map( + normalizedResources.map((resource) => [ + getResourceKey( + resource.type, + resource.id, + resource.type === 'model' ? resource.data.modelType : undefined + ), + resource + ]) + ); + const appIds = normalizedResources + .filter((resource) => resource.type === 'agent' || resource.type === 'tool') + .map((resource) => resource.id); + const datasetIds = normalizedResources + .filter((resource) => resource.type === 'dataset') + .map((resource) => resource.id); + const skillIds = normalizedResources + .filter((resource) => resource.type === 'skill') + .map((resource) => resource.id); + + const [apps, datasets, skills] = await Promise.all([ + appIds.length + ? MongoApp.find({ + _id: { $in: appIds }, + deleteTime: null, + ...(teamId && !isRoot ? { teamId } : {}) + }).lean() + : [], + datasetIds.length + ? MongoDataset.find({ + _id: { $in: datasetIds }, + deleteTime: null, + ...(teamId && !isRoot ? { teamId } : {}) + }).lean() + : [], + skillIds.length + ? MongoAgentSkills.find({ + _id: { $in: skillIds }, + deleteTime: null, + ...(teamId && !isRoot + ? { + $or: [{ teamId }, { source: AgentSkillSourceEnum.system }] + } + : {}) + }).lean() + : [] + ]); + + const appMap = new Map(apps.map((app) => [String(app._id), app])); + const workflowMap = apps.length + ? await getAppPublishedWorkflowMap(apps) + : new Map(); + const datasetMap = new Map(datasets.map((dataset) => [String(dataset._id), dataset])); + const skillMap = new Map(skills.map((skill) => [String(skill._id), skill])); + + return { + teamId, + isRoot, + appMap, + workflowMap, + datasetMap, + skillMap, + resources: normalizedResources, + resourceMap + } satisfies WorkflowResourceContext; +}; + +/** 将资源上下文转换为权限校验可复用的实体集合,避免重复查询资源实体。 */ +export const getWorkflowResourceEntities = ( + context: WorkflowResourceContext +): WorkflowResourceEntities => ({ + apps: Array.from(context.appMap.values()), + datasets: Array.from(context.datasetMap.values()), + skills: Array.from(context.skillMap.values()) +}); + +/** 校验当前工作流版本声明了指定资源;没有上下文时保留非 App 调试场景的旧权限语义。 */ +export const assertWorkflowResource = ({ + context, + type, + id, + toolName +}: { + context?: WorkflowResourceContext; + type: AppResourceType; + id: string; + toolName?: string; +}) => { + if (!context) return; + + const resource = context.resourceMap.get(getResourceKey(type, id)); + if (!resource) throw new WorkflowResourceError(`App resource is not declared: ${type}:${id}`); + + if (resource.type === 'tool' && toolName && resource.data?.toolNames?.length) { + if (!resource.data.toolNames.includes(toolName)) { + throw new WorkflowResourceError(`App tool is not declared: ${id}/${toolName}`); + } + } +}; + +/** 过滤工具集子工具;资源快照未限制子工具时返回完整工具集。 */ +export const filterWorkflowToolList = ({ + context, + appId, + tools +}: { + context?: WorkflowResourceContext; + appId: string; + tools: Tool[]; +}) => { + if (!context) return tools; + + const resource = context.resourceMap.get(getResourceKey('tool', appId)); + if (!resource || resource.type !== 'tool') { + throw new WorkflowResourceError(`App resource is not declared: tool:${appId}`); + } + + const toolNames = resource.data?.toolNames; + if (!toolNames?.length) return tools; + + const allowedNames = new Set(toolNames); + return tools.filter((tool) => allowedNames.has(tool.name)); +}; + +/** + * 校验工作流本次使用的知识库集合是否属于当前版本快照。 + * 动态输入不走快照,由调用方按运行人 tmbId 鉴权。 + */ +export const assertWorkflowDatasetResources = ({ + datasetIds, + dynamic = false +}: { + datasetIds: string[]; + dynamic?: boolean; +}) => { + const context = getWorkflowResourceContext(); + if (!context || dynamic) return; + + datasetIds.forEach((id) => { + assertWorkflowResource({ + context, + type: 'dataset', + id + }); + if (!context.datasetMap.has(id)) throw DatasetErrEnum.unExist; + }); +}; + +/** 读取当前资源上下文已批量加载的知识库实体。 */ +export const getWorkflowDatasetResource = (datasetId: string) => + getWorkflowResourceContext()?.datasetMap.get(datasetId); + +/** 读取当前资源上下文中 App 对应的正式工作流。 */ +export const getWorkflowAppWorkflow = (appId: string) => + getWorkflowResourceContext()?.workflowMap?.get(appId); + +/** + * 读取工作流使用的知识库。 + * 静态引用必须命中当前 Version 快照;动态引用按运行人 tmbId 鉴权。 + * 没有 resourceContext 时(Skill 调试、商业工具清空父快照)也按运行人鉴权,不能裸 findById。 + */ +export const loadWorkflowDatasetResource = async ({ + datasetId, + datasetIds = [datasetId], + dynamic = false, + tmbId +}: { + datasetId: string; + /** 静态工作流可能一次使用多个知识库,加载首个实体前需要完整校验声明集合。 */ + datasetIds?: string[]; + dynamic?: boolean; + tmbId?: string; +}) => { + const context = getWorkflowResourceContext(); + if (context && !dynamic) { + assertWorkflowDatasetResources({ datasetIds }); + const dataset = context.datasetMap.get(datasetId); + if (!dataset) throw DatasetErrEnum.unExist; + return dataset; + } + + if (tmbId) { + return ( + await authDatasetByTmbId({ + tmbId, + datasetId, + per: ReadPermissionVal + }) + ).dataset; + } + + return Promise.reject(DatasetErrEnum.unExist); +}; + +/** + * 加载 App 或工具集。 + * 静态引用命中当前 Version 快照;动态引用或没有 resourceContext 时按运行人读权限查询。 + */ +export const loadWorkflowAppResource = async ({ + appId, + tmbId, + type, + toolName, + dynamic = false +}: { + appId: string; + tmbId: string; + type: Extract; + toolName?: string; + dynamic?: boolean; +}) => { + const context = getWorkflowResourceContext(); + if (!context || dynamic) { + return ( + await authAppByTmbId({ + appId, + tmbId, + per: ReadPermissionVal + }) + ).app; + } + + assertWorkflowResource({ context, type, id: appId, toolName }); + const app = context.appMap.get(appId); + if (!app) throw AppErrEnum.unExist; + return app; +}; + +/** 为子 App 创建独立快照;父子 App 不共享资源声明,但继承 root 调试请求的跨团队权限。 */ +export const createWorkflowChildResourceContext = ( + resources: AppResourcesType, + teamId?: string, + isRoot = getWorkflowResourceContext()?.isRoot ?? false +) => loadWorkflowResourceContext({ resources, teamId, isRoot }); diff --git a/packages/service/support/outLink/runtime/service.ts b/packages/service/support/outLink/runtime/service.ts index 9e2ce814b88a..140d85f3bc43 100644 --- a/packages/service/support/outLink/runtime/service.ts +++ b/packages/service/support/outLink/runtime/service.ts @@ -25,6 +25,7 @@ import { } from '@fastgpt/global/core/chat/utils'; import type { OutlinkAppType, OutLinkSchemaType } from '@fastgpt/global/support/outLink/type'; import { getAppLatestVersion } from '../../../core/app/version/controller'; +import { loadWorkflowResourceContext } from '../../../core/workflow/utils/resource'; import { MongoApp } from '../../../core/app/schema'; import { getChatItems } from '../../../core/chat/controller'; import { @@ -270,10 +271,11 @@ export async function runOutlinkRuntime({ try { // Load the published app and its latest workflow config in parallel. - const [app, { nodes, chatConfig, edges }] = await Promise.all([ + const [app, workflowVersion] = await Promise.all([ MongoApp.findById(outLinkConfig.appId).lean(), getAppLatestVersion(outLinkConfig.appId) ]); + const { nodes, chatConfig, edges, resources } = workflowVersion; if (!nodes || !chatConfig || !app) { return Promise.reject('Invalid chat'); @@ -407,6 +409,10 @@ export async function runOutlinkRuntime({ uid: chatUserId || outLinkConfig.tmbId, chatId: preparedRound.chatId, responseChatItemId: preparedRound.responseChatItemId, + resourceContext: await loadWorkflowResourceContext({ + resources, + teamId: app.teamId + }), variables, histories, query: workflowQuery, diff --git a/packages/service/support/permission/app/auth.ts b/packages/service/support/permission/app/auth.ts index ec3194a5c47d..f46f895e6eac 100644 --- a/packages/service/support/permission/app/auth.ts +++ b/packages/service/support/permission/app/auth.ts @@ -1,6 +1,6 @@ /* Auth app permission */ import { MongoApp } from '../../../core/app/schema'; -import { type AppDetailType } from '@fastgpt/global/core/app/type'; +import { type AppWithPermissionType } from '@fastgpt/global/core/app/type'; import { PerResourceTypeEnum, ReadPermissionVal, @@ -50,7 +50,7 @@ export const authAppByTmbId = async ({ per: PermissionValueType; isRoot?: boolean; }): Promise<{ - app: AppDetailType; + app: AppWithPermissionType; }> => { const { teamId, permission: tmbPer } = await getTmbInfoByTmbId({ tmbId }); @@ -141,7 +141,7 @@ export const authApp = async ({ per: PermissionValueType; }): Promise< AuthResponseType & { - app: AppDetailType; + app: AppWithPermissionType; } > => { const result = await parseHeaderCert(props); diff --git a/packages/service/support/permission/app/resource.ts b/packages/service/support/permission/app/resource.ts new file mode 100644 index 000000000000..b87277054adb --- /dev/null +++ b/packages/service/support/permission/app/resource.ts @@ -0,0 +1,328 @@ +import type { AppResource } from '@fastgpt/global/core/app/type'; +import type { WorkflowResourceEntities } from '../../../core/workflow/utils/resource'; +import { AppFolderTypeList, AppTypeEnum } from '@fastgpt/global/core/app/constants'; +import { DatasetTypeEnum } from '@fastgpt/global/core/dataset/constants'; +import { AgentSkillSourceEnum, AgentSkillTypeEnum } from '@fastgpt/global/core/ai/skill/constants'; +import { + NullRoleVal, + PerResourceTypeEnum, + ReadRoleVal, + ReadPermissionVal +} from '@fastgpt/global/support/permission/constant'; +import { AppErrEnum } from '@fastgpt/global/common/error/code/app'; +import { DatasetErrEnum } from '@fastgpt/global/common/error/code/dataset'; +import { SkillErrEnum } from '@fastgpt/global/common/error/code/skill'; +import { AppPermission } from '@fastgpt/global/support/permission/app/controller'; +import { DatasetPermission } from '@fastgpt/global/support/permission/dataset/controller'; +import { SkillPermission } from '@fastgpt/global/support/permission/skill/controller'; +import { sumPer } from '@fastgpt/global/support/permission/utils'; +import { MongoApp } from '../../../core/app/schema'; +import { MongoDataset } from '../../../core/dataset/schema'; +import { MongoAgentSkills } from '../../../core/ai/skill/model/schema'; +import { MongoResourcePermission } from '../schema'; +import type { ClientSession } from '../../../common/mongo'; +import { getTmbInfoByTmbId } from '../../user/team/controller'; +import { getGroupsByTmbId } from '../memberGroup/controllers'; +import { getOrgIdSetWithParentByTmbId } from '../org/controllers'; +import { shouldInheritResourcePermission } from '../resourcePermissionPolicy'; +import { + getAppResourceKey, + mergeAppResources, + splitExtractedAppResources +} from '../../../core/app/resources'; +import { getAppDraftResourceBaseline } from '../../../core/app/version/controller'; + +const getResourcePermission = ({ + permissions, + id, + parentId, + inheritPermission, + canInherit +}: { + permissions: Map; + id: string; + parentId?: string; + inheritPermission: boolean; + canInherit: boolean; +}) => + sumPer( + permissions.get(id) ?? NullRoleVal, + ...(inheritPermission && canInherit && parentId + ? [permissions.get(parentId) ?? NullRoleVal] + : []) + ); + +/** 返回当前操作人没有读取权限(或不存在)的资源,不抛错。 */ +export const getUnauthorizedAppResources = async ({ + resources, + tmbId, + isRoot = false, + resourceEntities, + allowRootCrossTeam = false +}: { + resources: AppResource[]; + tmbId: string; + isRoot?: boolean; + resourceEntities?: WorkflowResourceEntities; + /** 仅 Test/Debug 临时上下文允许 root 校验跨团队资源。 */ + allowRootCrossTeam?: boolean; +}) => { + const appIds = Array.from( + new Set( + resources + .filter((resource) => resource.type === 'agent' || resource.type === 'tool') + .map((resource) => resource.id) + ) + ); + const datasetIds = Array.from( + new Set( + resources.filter((resource) => resource.type === 'dataset').map((resource) => resource.id) + ) + ); + const skillIds = Array.from( + new Set( + resources.filter((resource) => resource.type === 'skill').map((resource) => resource.id) + ) + ); + + if (appIds.length === 0 && datasetIds.length === 0 && skillIds.length === 0) return []; + + const [{ teamId, permission: tmbPermission }, loadedEntities] = await Promise.all([ + getTmbInfoByTmbId({ tmbId }), + resourceEntities ?? + Promise.all([ + appIds.length + ? MongoApp.find({ _id: { $in: appIds }, deleteTime: null }).lean() + : Promise.resolve([]), + datasetIds.length + ? MongoDataset.find({ _id: { $in: datasetIds }, deleteTime: null }).lean() + : Promise.resolve([]), + skillIds.length + ? MongoAgentSkills.find({ _id: { $in: skillIds }, deleteTime: null }).lean() + : Promise.resolve([]) + ]).then(([apps, datasets, skills]) => ({ apps, datasets, skills })) + ]); + const { apps, datasets, skills } = loadedEntities; + + const [groupList, orgIdSet] = await Promise.all([ + getGroupsByTmbId({ tmbId, teamId }), + getOrgIdSetWithParentByTmbId({ tmbId, teamId }) + ]); + const groupIds = groupList.map((group) => String(group._id)); + const orgIds = Array.from(orgIdSet); + + const appResourceIds = Array.from( + new Set( + apps.flatMap((app) => [String(app._id), ...(app.parentId ? [String(app.parentId)] : [])]) + ) + ); + const datasetResourceIds = Array.from( + new Set( + datasets.flatMap((dataset) => [ + String(dataset._id), + ...(dataset.parentId ? [String(dataset.parentId)] : []) + ]) + ) + ); + const skillResourceIds = Array.from( + new Set( + skills.flatMap((skill) => [ + String(skill._id), + ...(skill.parentId ? [String(skill.parentId)] : []) + ]) + ) + ); + + const permissionList = await MongoResourcePermission.find( + { + teamId, + resourceType: { + $in: [PerResourceTypeEnum.app, PerResourceTypeEnum.dataset, PerResourceTypeEnum.agentSkill] + }, + resourceId: { $in: [...appResourceIds, ...datasetResourceIds, ...skillResourceIds] }, + $or: [ + { tmbId }, + ...(groupIds.length ? [{ groupId: { $in: groupIds } }] : []), + ...(orgIds.length ? [{ orgId: { $in: orgIds } }] : []) + ] + }, + 'resourceType resourceId permission tmbId' + ).lean(); + + const permissionMap = new Map(); + permissionList.forEach((item) => { + const key = `${item.resourceType}:${String(item.resourceId)}`; + const record = permissionMap.get(key) ?? { inherited: [] }; + if (item.tmbId) { + if (record.direct === undefined) record.direct = item.permission; + } else { + record.inherited.push(item.permission); + } + permissionMap.set(key, record); + }); + + const buildPermissionMap = (resourceType: PerResourceTypeEnum, resourceIds: string[]) => + new Map( + resourceIds.map((resourceId) => { + const record = permissionMap.get(`${resourceType}:${resourceId}`); + return [resourceId, record?.direct ?? sumPer(...(record?.inherited ?? []))]; + }) + ); + + const appPermissions = buildPermissionMap(PerResourceTypeEnum.app, appResourceIds); + const datasetPermissions = buildPermissionMap(PerResourceTypeEnum.dataset, datasetResourceIds); + const skillPermissions = buildPermissionMap(PerResourceTypeEnum.agentSkill, skillResourceIds); + + const appMap = new Map(apps.map((app) => [String(app._id), app])); + const datasetMap = new Map(datasets.map((dataset) => [String(dataset._id), dataset])); + const skillMap = new Map(skills.map((skill) => [String(skill._id), skill])); + + const unauthorized: { resource: AppResource; error: unknown }[] = []; + const reject = (resource: AppResource, error: unknown) => { + unauthorized.push({ resource, error }); + }; + + resources.forEach((resource) => { + if (resource.type === 'agent' || resource.type === 'tool') { + const app = appMap.get(resource.id); + if (!app) { + reject(resource, AppErrEnum.unExist); + return; + } + if (!allowRootCrossTeam && String(app.teamId) !== teamId) { + reject(resource, AppErrEnum.unAuthApp); + return; + } + if (isRoot) return; + if (app.type === AppTypeEnum.hidden) return; + + const permission = new AppPermission({ + role: getResourcePermission({ + permissions: appPermissions, + id: resource.id, + parentId: app.parentId ? String(app.parentId) : undefined, + inheritPermission: shouldInheritResourcePermission(app.inheritPermission), + canInherit: !AppFolderTypeList.includes(app.type) + }), + isOwner: tmbPermission.isOwner || String(app.tmbId) === tmbId + }); + if (app.favourite || app.quick) permission.addRole(ReadRoleVal); + if (!permission.checkPer(ReadPermissionVal)) reject(resource, AppErrEnum.unAuthApp); + return; + } + + if (resource.type === 'dataset') { + const dataset = datasetMap.get(resource.id); + if (!dataset) { + reject(resource, DatasetErrEnum.unExist); + return; + } + if (!allowRootCrossTeam && String(dataset.teamId) !== teamId) { + reject(resource, DatasetErrEnum.unAuthDataset); + return; + } + if (isRoot) return; + + const permission = new DatasetPermission({ + role: getResourcePermission({ + permissions: datasetPermissions, + id: resource.id, + parentId: dataset.parentId ? String(dataset.parentId) : undefined, + inheritPermission: shouldInheritResourcePermission(dataset.inheritPermission), + canInherit: dataset.type !== DatasetTypeEnum.folder + }), + isOwner: tmbPermission.isOwner || String(dataset.tmbId) === tmbId + }); + if (!permission.checkPer(ReadPermissionVal)) reject(resource, DatasetErrEnum.unAuthDataset); + return; + } + + if (resource.type === 'skill') { + const skill = skillMap.get(resource.id); + if (!skill) { + reject(resource, SkillErrEnum.unExist); + return; + } + if (skill.source === AgentSkillSourceEnum.system) return; + if (!allowRootCrossTeam && String(skill.teamId) !== teamId) { + reject(resource, SkillErrEnum.unAuthSkill); + return; + } + if (isRoot) return; + + const permission = new SkillPermission({ + role: getResourcePermission({ + permissions: skillPermissions, + id: resource.id, + parentId: skill.parentId ? String(skill.parentId) : undefined, + inheritPermission: skill.inheritPermission !== false, + canInherit: skill.type !== AgentSkillTypeEnum.folder + }), + isOwner: tmbPermission.isOwner || String(skill.tmbId) === tmbId + }); + if (!permission.checkPer(ReadPermissionVal)) reject(resource, SkillErrEnum.unAuthSkill); + } + }); + + return unauthorized; +}; + +/** 在保存、发布或 Test/Debug 边界批量校验应用引用资源的读取权限。 */ +export const checkAppResourceReadPermissions = async ( + props: Parameters[0] +) => { + const unauthorized = await getUnauthorizedAppResources(props); + if (unauthorized[0]) throw unauthorized[0].error; +}; + +/** + * 按当前应用草稿快照解析资源,并只校验相对快照新增的 ACL 资源。 + * 保存/自动保存不阻断无权限新增;发布和 Test/Debug 通过 blockOnUnauthorized 阻断。 + * 保存/发布事务传入 session 时,baseline 会在事务内读取,事务重试会重新计算资源快照。 + */ +export const resolveAppResourcesByPermission = async ({ + appId, + extracted, + tmbId, + isRoot = false, + blockOnUnauthorized, + allowRootCrossTeam = false, + resourceEntities, + session +}: { + appId: string; + extracted: AppResource[]; + tmbId: string; + isRoot?: boolean; + blockOnUnauthorized: boolean; + allowRootCrossTeam?: boolean; + resourceEntities?: WorkflowResourceEntities; + session?: ClientSession; +}) => { + const baseline = await getAppDraftResourceBaseline(appId, session); + const { kept, added } = splitExtractedAppResources({ extracted, baseline }); + if (added.length === 0) return mergeAppResources(kept); + if (blockOnUnauthorized) { + await checkAppResourceReadPermissions({ + resources: added, + tmbId, + isRoot, + allowRootCrossTeam, + resourceEntities + }); + return mergeAppResources([...kept, ...added]); + } + + const unauthorized = await getUnauthorizedAppResources({ + resources: added, + tmbId, + isRoot, + allowRootCrossTeam, + resourceEntities + }); + const unauthorizedKeys = new Set(unauthorized.map((item) => getAppResourceKey(item.resource))); + return mergeAppResources([ + ...kept, + ...added.filter((resource) => !unauthorizedKeys.has(getAppResourceKey(resource))) + ]); +}; diff --git a/packages/service/support/permission/publish/authLink.ts b/packages/service/support/permission/publish/authLink.ts index f1dc55a70acc..2595ae25f8ab 100644 --- a/packages/service/support/permission/publish/authLink.ts +++ b/packages/service/support/permission/publish/authLink.ts @@ -1,4 +1,4 @@ -import { type AppDetailType } from '@fastgpt/global/core/app/type'; +import { type AppWithPermissionType } from '@fastgpt/global/core/app/type'; import { type OutlinkAppType, type OutLinkSchemaType } from '@fastgpt/global/support/outLink/type'; import { MongoOutLink } from '../../outLink/schema'; import { OutLinkErrEnum } from '@fastgpt/global/common/error/code/outLink'; @@ -23,7 +23,7 @@ export async function authOutLinkCrud({ outLinkId: string; }): Promise< AuthResponseType & { - app: AppDetailType; + app: AppWithPermissionType; outLink: OutLinkSchemaType; } > { diff --git a/packages/service/support/permission/skill/auth.ts b/packages/service/support/permission/skill/auth.ts index 3df87a31c4b3..e811d40e0533 100644 --- a/packages/service/support/permission/skill/auth.ts +++ b/packages/service/support/permission/skill/auth.ts @@ -54,11 +54,7 @@ export const authSkillByTmbId = async ({ }; } - if (String(skill.teamId) !== teamId) { - return Promise.reject(SkillErrEnum.unAuthSkill); - } - - // System skills are read-only for all team members + // System skills are global read-only resources and do not belong to a team. if (skill.source === AgentSkillSourceEnum.system) { const sysPer = new SkillPermission({ role: ReadRoleVal, isOwner: false }); if (!sysPer.checkPer(per)) { @@ -70,6 +66,11 @@ export const authSkillByTmbId = async ({ }; } + if (String(skill.teamId) !== teamId) { + return Promise.reject(SkillErrEnum.unAuthSkill); + } + + // Check if should inherit permission from parent folder const isOwner = tmbPer.isOwner || String(skill.tmbId) === String(tmbId); const isGetParentClb = shouldInheritResourcePermission(skill.inheritPermission) && diff --git a/packages/service/test/core/ai/sandbox/application/runtime/skill/core.test.ts b/packages/service/test/core/ai/sandbox/application/runtime/skill/core.test.ts index ff51d3d40298..79ac76ce3153 100644 --- a/packages/service/test/core/ai/sandbox/application/runtime/skill/core.test.ts +++ b/packages/service/test/core/ai/sandbox/application/runtime/skill/core.test.ts @@ -10,6 +10,8 @@ import { uploadSkillPackage } from '@fastgpt/service/core/ai/skill/package'; import { AgentSkillSourceEnum } from '@fastgpt/global/core/ai/skill/constants'; import { Types } from '@fastgpt/service/common/mongo'; import { getNanoid } from '@fastgpt/global/common/string/tools'; +import { runWithContext } from '@fastgpt/service/core/workflow/utils/context'; +import { loadWorkflowResourceContext } from '@fastgpt/service/core/workflow/utils/resource'; import { getUser } from '@test/datas/users'; import { PerResourceTypeEnum, @@ -444,6 +446,73 @@ description: Latest current skill ]); }); + it('injects a global system skill from the resource snapshot', async () => { + const user = await getUser(`runtime-system-skill-${getNanoid(6)}`); + const { teamId, tmbId } = user; + const skill = await MongoAgentSkills.create({ + name: 'GlobalSystemSkill', + description: '', + teamId: null, + tmbId: null, + source: AgentSkillSourceEnum.system + }); + const versionId = new Types.ObjectId(); + const skillPackage = await makePackage([ + { path: 'skill.md', name: 'system', description: 'Global system skill' } + ]); + const storage = await uploadSkillPackage({ + teamId, + skillId: String(skill._id), + packageObjectId: 'runtime-system-version', + zipBuffer: skillPackage + }); + await MongoAgentSkillsVersion.create({ + _id: versionId, + skillId: skill._id, + tmbId, + storageKey: storage.key + }); + await MongoAgentSkills.updateOne({ _id: skill._id }, { $set: { currentVersionId: versionId } }); + + const targetDir = `/workspace/projects/${String(versionId)}`; + const sandbox = { + ...createSkillFilesystemMocks(), + writeFiles: vi.fn(async (entries: Array<{ path: string; data: Buffer }>) => + makeWriteResults(entries) + ), + execute: vi.fn(async (command: string) => { + if (command.includes('unzip')) return { exitCode: 0, stdout: '', stderr: '' }; + throw new Error(`Unexpected command: ${command}`); + }), + readFiles: vi.fn() + }; + + const resourceContext = await loadWorkflowResourceContext({ + resources: [{ type: 'skill', id: String(skill._id) }], + teamId + }); + const staticVersions = await runWithContext({ mcpClientMemory: {}, resourceContext }, () => + injectAgentSkillFilesToSandbox({ + sandbox: sandbox as any, + skillIds: [String(skill._id)], + teamId, + tmbId, + workDirectory: '/workspace' + }) + ); + + expect(staticVersions).toEqual([ + { + skillId: String(skill._id), + name: 'GlobalSystemSkill', + description: '', + avatar: undefined, + versionId: String(versionId), + targetDir + } + ]); + }); + it('filters unauthorized runtime skills instead of injecting them', async () => { const owner = await getUser(`runtime-skill-owner-${getNanoid(6)}`); const runner = await getUser(`runtime-skill-runner-${getNanoid(6)}`, owner.teamId); diff --git a/packages/service/test/core/app/http.test.ts b/packages/service/test/core/app/http.test.ts index 9121d32b1699..d4ca6da695e6 100644 --- a/packages/service/test/core/app/http.test.ts +++ b/packages/service/test/core/app/http.test.ts @@ -147,30 +147,31 @@ describe('SSRF Vulnerability Fix Tests', () => { }); describe('getHTTPToolList', () => { + const createHttpWorkflow = (toolSet: Record) => ({ + nodes: [{ toolConfig: { httpToolSet: toolSet } }] + }); + it('preserves stored schemas when legacy OpenAPI text cannot be parsed', async () => { const requestSchema = { type: 'object', properties: { q: { type: 'string' } } }; - const [tool] = await getHTTPToolList({ - _id: 'http-toolset', - type: AppTypeEnum.httpToolSet, - modules: [ + const toolSet = { + apiSchemaStr: '{"openapi":"3.1.0"}', + toolList: [ { - toolConfig: { - httpToolSet: { - apiSchemaStr: '{"openapi":"3.1.0"}', - toolList: [ - { - name: 'search', - description: 'Search', - path: '/search', - method: 'GET', - requestSchema - } - ] - } - } + name: 'search', + description: 'Search', + path: '/search', + method: 'GET', + requestSchema } ] - } as any); + }; + const [tool] = await getHTTPToolList( + { + _id: '507f1f77bcf86cd799439011', + type: AppTypeEnum.httpToolSet + } as any, + createHttpWorkflow(toolSet) as any + ); expect(tool.requestSchema).toEqual(requestSchema); expect(tool.requestSchema).not.toHaveProperty('required'); }); @@ -203,81 +204,64 @@ describe('SSRF Vulnerability Fix Tests', () => { } }); const app = { - _id: 'http-toolset', + _id: '507f1f77bcf86cd799439011', type: AppTypeEnum.httpToolSet, - modules: [ + name: 'HTTP tools' + }; + const toolSet = { + apiSchemaStr, + toolList: [ { - toolConfig: { - httpToolSet: { - apiSchemaStr, - toolList: [ - { - name: 'echo', - description: 'Echo', - path: '/echo/{id}', - method: 'POST', - inputSchema, - requestSchema - } - ] - } - } + name: 'echo', + description: 'Echo', + path: '/echo/{id}', + method: 'POST', + inputSchema, + requestSchema } ] }; - const [tool] = await getHTTPToolList(app as any); + const [tool] = await getHTTPToolList(app as any, createHttpWorkflow(toolSet) as any); expect(tool.requestSchema).toEqual({ ...requestSchema, properties: { q: { type: 'string' }, id: { type: 'string' }, ...requestSchema.properties }, required: ['body', 'id'] }); expect(tool.inputSchema).toEqual(inputSchema); - expect(app.modules[0].toolConfig.httpToolSet.toolList[0].requestSchema).toEqual( - requestSchema - ); + expect(toolSet.toolList[0].requestSchema).toEqual(requestSchema); expect( ( - await getHTTPToolList({ - ...app, - modules: [ - { - toolConfig: { - httpToolSet: { ...app.modules[0].toolConfig.httpToolSet, apiSchemaStr: undefined } - } - } - ] - } as any) + await getHTTPToolList( + app as any, + createHttpWorkflow({ ...toolSet, apiSchemaStr: undefined }) as any + ) )[0].requestSchema ).toEqual(requestSchema); }); it('should read tools when legacy customHeaders has a non-string value', async () => { - const result = await getHTTPToolList({ - _id: 'http-toolset', - type: AppTypeEnum.httpToolSet, - modules: [ - { - toolConfig: { - httpToolSet: { - customHeaders: false, - toolList: [ - { - name: 'search', - description: 'Search', - path: '/search', - method: 'GET' - } - ] - } + const result = await getHTTPToolList( + { + _id: '507f1f77bcf86cd799439011', + type: AppTypeEnum.httpToolSet + } as any, + createHttpWorkflow({ + customHeaders: false, + toolList: [ + { + name: 'search', + description: 'Search', + path: '/search', + method: 'GET' } - } - ] - } as any); + ] + }) as any + ); expect(result).toMatchObject([ { name: 'search', - id: 'http-http-toolset/search' + id: 'http-507f1f77bcf86cd799439011/search' } ]); }); diff --git a/packages/service/test/core/app/mcp.test.ts b/packages/service/test/core/app/mcp.test.ts index e6647c6791a7..61d8933f90cf 100644 --- a/packages/service/test/core/app/mcp.test.ts +++ b/packages/service/test/core/app/mcp.test.ts @@ -3,9 +3,8 @@ import http from 'http'; import os from 'os'; // --- Hoisted mocks --- -const { mockDereference, mockMongoAppFind } = vi.hoisted(() => ({ - mockDereference: vi.fn(), - mockMongoAppFind: vi.fn() +const { mockDereference } = vi.hoisted(() => ({ + mockDereference: vi.fn() })); vi.mock('@apidevtools/json-schema-ref-parser', () => ({ @@ -14,12 +13,6 @@ vi.mock('@apidevtools/json-schema-ref-parser', () => ({ } })); -vi.mock('../../../core/app/schema', () => ({ - MongoApp: { - find: mockMongoAppFind - } -})); - import { StreamableHTTPError } from '@modelcontextprotocol/sdk/client/streamableHttp.js'; import { MCPClient, @@ -29,9 +22,10 @@ import { } from '../../../core/app/mcp'; import type { AppSchemaType } from '@fastgpt/global/core/app/type'; import { AppTypeEnum } from '@fastgpt/global/core/app/constants'; +import { FlowNodeTypeEnum } from '@fastgpt/global/core/workflow/node/constant'; +import type { AppPublishedWorkflow } from '../../../core/app/version/controller'; import { PRIVATE_URL_TEXT } from '../../../common/system/utils'; import { serviceEnv } from '../../../env'; -import { getSecretValue, storeSecretValue } from '../../../common/secret/utils'; // Access private client via prototype for spying const getPrivateClient = (mcpClient: MCPClient) => @@ -687,39 +681,47 @@ describe('createMcpSafeFetch', () => { }); describe('getMCPChildren', () => { + const createMcpWorkflow = ( + toolConfig?: NonNullable + ): AppPublishedWorkflow => ({ + nodes: [ + { + nodeId: 'mcp-toolset', + flowNodeType: FlowNodeTypeEnum.toolSet, + name: 'MCP toolset', + toolConfig, + inputs: [], + outputs: [] + } + ] + }); + it('should return tool list from new MCP format', async () => { const app = { _id: 'app123', avatar: '/icon.png', teamId: 'team1', - type: AppTypeEnum.mcpToolSet, - modules: [ - { - toolConfig: { - mcpToolSet: { - toolId: 'tid', - url: 'http://mcp.test', - toolList: [ - { - name: 'tool_a', - description: 'A', - inputSchema: { type: 'object', properties: {} } - }, - { - name: 'tool_b', - description: 'B', - inputSchema: { type: 'object', properties: {} } - } - ] - } + type: AppTypeEnum.mcpToolSet + } as AppSchemaType; + const workflow = createMcpWorkflow({ + mcpToolSet: { + url: 'http://mcp.test', + toolList: [ + { + name: 'tool_a', + description: 'A', + inputSchema: { type: 'object', properties: {} } }, - inputs: [], - outputs: [] - } - ] - } as unknown as AppSchemaType; + { + name: 'tool_b', + description: 'B', + inputSchema: { type: 'object', properties: {} } + } + ] + } + }); - const result = await getMCPChildren(app); + const result = await getMCPChildren(app, workflow); expect(result).toHaveLength(2); expect(result[0]).toMatchObject({ @@ -739,215 +741,16 @@ describe('getMCPChildren', () => { _id: 'app123', avatar: '/icon.png', teamId: 'team1', - type: AppTypeEnum.mcpToolSet, - modules: [ - { - toolConfig: { - mcpToolSet: { - toolId: 'tid', - url: 'http://mcp.test', - toolList: [] - } - }, - inputs: [], - outputs: [] - } - ] - } as unknown as AppSchemaType; - - const result = await getMCPChildren(app); - expect(result).toEqual([]); - }); - - it.each([ - { headerSecret: undefined, expected: undefined, headers: {} }, - { headerSecret: null, expected: undefined, headers: {} }, - { headerSecret: {}, expected: {}, headers: {} }, - { - headerSecret: { value: 'legacy-token' }, - expected: { Authorization: { value: 'legacy-token' } }, - headers: { Authorization: 'legacy-token' } - }, - { - headerSecret: { Authorization: { value: 'legacy-token' }, 'X-Key': { value: 'api-key' } }, - expected: { Authorization: { value: 'legacy-token' }, 'X-Key': { value: 'api-key' } }, - headers: { Authorization: 'legacy-token', 'X-Key': 'api-key' } - }, - { - headerSecret: { value: { value: 'header-named-value' } }, - expected: { value: { value: 'header-named-value' } }, - headers: { value: 'header-named-value' } - } - ])( - 'normalizes legacy child headers without wrapping an existing map: %j', - async ({ headerSecret, expected, headers }) => { - const children = [ - { - name: 'search', - modules: [ - { - inputs: [ - { - value: { - name: 'search', - description: 'Search', - url: 'https://mcp.example.com', - headerSecret, - inputSchema: { type: 'object' } - } - } - ] - } - ] - } - ]; - const original = structuredClone(children); - mockMongoAppFind.mockReturnValueOnce({ lean: async () => children }); - const [tool] = await getMCPChildren({ - _id: 'legacy-set', - teamId: 'team', - type: AppTypeEnum.mcpToolSet, - avatar: '', - modules: [{ inputs: [] }] - } as unknown as AppSchemaType); - expect(tool.headerSecret).toEqual(expected); - expect(getSecretValue({ storeSecret: tool.headerSecret })).toEqual(headers); - expect(children).toEqual(original); - } - ); - - it.each(['map', 'single'] as const)( - 'keeps encrypted legacy %s headers decryptable', - async (shape) => { - const encrypted = storeSecretValue({ - Authorization: { value: 'legacy-token' }, - 'X-Key': { value: 'api-key' } - }); - const headerSecret = shape === 'map' ? encrypted : encrypted.Authorization; - mockMongoAppFind.mockReturnValueOnce({ - lean: async () => [ - { - name: 'search', - modules: [ - { - inputs: [ - { - value: { - name: 'search', - description: 'Search', - url: 'https://mcp.example.com', - headerSecret - } - } - ] - } - ] - } - ] - }); - const [tool] = await getMCPChildren({ - _id: 'legacy-set', - teamId: 'team', - type: AppTypeEnum.mcpToolSet, - avatar: '', - modules: [{ inputs: [] }] - } as unknown as AppSchemaType); - expect(tool.headerSecret).toEqual( - shape === 'map' ? encrypted : { Authorization: encrypted.Authorization } - ); - expect(getSecretValue({ storeSecret: tool.headerSecret })).toEqual( - shape === 'map' - ? { Authorization: 'legacy-token', 'X-Key': 'api-key' } - : { Authorization: 'legacy-token' } - ); - } - ); - - it('should query MongoApp for old MCP format', async () => { - const app = { - _id: 'app456', - avatar: '/old-icon.png', - teamId: 'team2', - type: AppTypeEnum.mcpToolSet, - modules: [ - { - toolConfig: undefined, - inputs: [], - outputs: [] - } - ] - } as unknown as AppSchemaType; - - const childApps = [ - { - name: 'child_tool', - modules: [ - { - inputs: [ - { - value: { - name: 'child_tool', - description: 'child desc', - url: 'http://child.mcp', - inputSchema: { type: 'object', properties: {} } - } - } - ] - } - ] + type: AppTypeEnum.mcpToolSet + } as AppSchemaType; + const workflow = createMcpWorkflow({ + mcpToolSet: { + url: 'http://mcp.test', + toolList: [] } - ]; - mockMongoAppFind.mockReturnValue({ lean: () => Promise.resolve(childApps) }); - - const result = await getMCPChildren(app); - - expect(mockMongoAppFind).toHaveBeenCalledWith({ teamId: 'team2', parentId: 'app456' }); - expect(result).toHaveLength(1); - expect(result[0]).toMatchObject({ - avatar: '/old-icon.png', - id: 'mcp-app456/child_tool', - name: 'child_tool' }); - }); - - it('should return empty array for old MCP with no children', async () => { - const app = { - _id: 'app789', - avatar: '/icon.png', - teamId: 'team3', - type: AppTypeEnum.mcpToolSet, - modules: [{ toolConfig: undefined, inputs: [], outputs: [] }] - } as unknown as AppSchemaType; - - mockMongoAppFind.mockReturnValue({ lean: () => Promise.resolve([]) }); - - const result = await getMCPChildren(app); - expect(result).toEqual([]); - }); - - it('should ignore a non-MCP app even when its modules contain an MCP config', async () => { - const app = { - _id: 'workflow-app', - avatar: '/icon.png', - teamId: 'team3', - type: AppTypeEnum.workflow, - modules: [ - { - toolConfig: { - mcpToolSet: { - url: 'https://mcp.test', - toolList: [] - } - }, - inputs: [], - outputs: [] - } - ] - } as unknown as AppSchemaType; - - const result = await getMCPChildren(app); + const result = await getMCPChildren(app, workflow); expect(result).toEqual([]); - expect(mockMongoAppFind).not.toHaveBeenCalled(); }); }); diff --git a/packages/service/test/core/app/resourceLookup.test.ts b/packages/service/test/core/app/resourceLookup.test.ts new file mode 100644 index 000000000000..93a46d1a5f4e --- /dev/null +++ b/packages/service/test/core/app/resourceLookup.test.ts @@ -0,0 +1,89 @@ +import { beforeEach, describe, expect, it } from 'vitest'; +import { Types } from '@fastgpt/service/common/mongo'; +import { MongoApp } from '@fastgpt/service/core/app/schema'; +import { MongoAppVersion } from '@fastgpt/service/core/app/version/schema'; +import { findTeamAppsByPublishedResource } from '@fastgpt/service/core/app/resourceLookup'; + +const teamId = new Types.ObjectId('65f000000000000000000071'); +const otherTeamId = new Types.ObjectId('65f000000000000000000072'); +const tmbId = new Types.ObjectId('65f000000000000000000073'); +const appId = new Types.ObjectId('65f000000000000000000074'); +const otherAppId = new Types.ObjectId('65f000000000000000000075'); +const publishedVersionId = new Types.ObjectId('65f000000000000000000076'); +const oldVersionId = new Types.ObjectId('65f000000000000000000077'); +const otherTeamVersionId = new Types.ObjectId('65f000000000000000000078'); + +describe('findTeamAppsByPublishedResource', () => { + beforeEach(async () => { + await Promise.all([MongoApp.deleteMany({}), MongoAppVersion.deleteMany({})]); + }); + + it('only counts resources from the current published version in the same team', async () => { + await MongoApp.collection.insertMany([ + { + _id: appId, + teamId, + tmbId, + name: 'Current app', + type: 'workflow', + publishedVersionId, + deleteTime: null + }, + { + _id: otherAppId, + teamId: otherTeamId, + tmbId, + name: 'Other team app', + type: 'workflow', + publishedVersionId: otherTeamVersionId, + deleteTime: null + } + ]); + await MongoAppVersion.collection.insertMany([ + { + _id: oldVersionId, + appId, + tmbId, + time: new Date('2026-08-01T00:00:00.000Z'), + isPublish: true, + resources: [{ type: 'skill', id: 'removed-skill' }] + }, + { + _id: publishedVersionId, + appId, + tmbId, + time: new Date('2026-08-20T00:00:00.000Z'), + isPublish: true, + resources: [ + { type: 'skill', id: 'skill-1' }, + { type: 'skill', id: 'skill-1' } + ] + }, + { + _id: otherTeamVersionId, + appId: otherAppId, + tmbId, + time: new Date('2026-08-20T00:00:00.000Z'), + isPublish: true, + resources: [{ type: 'skill', id: 'skill-1' }] + } + ]); + + const { apps, counts } = await findTeamAppsByPublishedResource({ + teamId: String(teamId), + type: 'skill', + ids: 'skill-1', + projection: 'name' + }); + + expect(apps.map((app) => String(app._id))).toEqual([String(appId)]); + expect(apps[0]).toMatchObject({ name: 'Current app' }); + expect(counts.get('skill-1')).toBe(1); + const removed = await findTeamAppsByPublishedResource({ + teamId: String(teamId), + type: 'skill', + ids: 'removed-skill' + }); + expect(removed.apps).toHaveLength(0); + }); +}); diff --git a/packages/service/test/core/app/resources.test.ts b/packages/service/test/core/app/resources.test.ts new file mode 100644 index 000000000000..ddb2bc84894c --- /dev/null +++ b/packages/service/test/core/app/resources.test.ts @@ -0,0 +1,274 @@ +import { describe, expect, it } from 'vitest'; +import { NodeInputKeyEnum } from '@fastgpt/global/core/workflow/constants'; +import { + FlowNodeInputTypeEnum, + FlowNodeTypeEnum +} from '@fastgpt/global/core/workflow/node/constant'; +import type { StoreNodeItemType } from '@fastgpt/global/core/workflow/type/node'; +import { + extractAppResources, + mergeAppResources, + nodeHasDynamicInput, + resolveStoredAppResources, + splitExtractedAppResources +} from '@fastgpt/service/core/app/resources'; + +const createInput = (key: string, value: unknown, reference = false) => ({ + key, + label: key, + value, + renderTypeList: reference ? [FlowNodeInputTypeEnum.reference] : [FlowNodeInputTypeEnum.input] +}); + +const createNode = ({ + flowNodeType = FlowNodeTypeEnum.workflowStart, + pluginId, + inputs = [] +}: { + flowNodeType?: FlowNodeTypeEnum; + pluginId?: string; + inputs?: ReturnType[]; +}) => + ({ + nodeId: `${flowNodeType}-node`, + name: 'node', + flowNodeType, + pluginId, + inputs, + outputs: [] + }) as unknown as StoreNodeItemType; + +describe('extractAppResources', () => { + it('normalizes personal, MCP and HTTP tools into parent resources', () => { + const resources = extractAppResources({ + nodes: [ + createNode({ + flowNodeType: FlowNodeTypeEnum.tool, + pluginId: 'personal-tool-app' + }), + createNode({ + flowNodeType: FlowNodeTypeEnum.tool, + pluginId: 'mcp-mcp-app/search' + }), + createNode({ + flowNodeType: FlowNodeTypeEnum.tool, + pluginId: 'http-http-app/request' + }), + createNode({ + flowNodeType: FlowNodeTypeEnum.tool, + pluginId: 'systemTool-search' + }) + ] + }); + + expect(resources).toEqual([ + { type: 'tool', id: 'http-app', data: { toolNames: ['request'] } }, + { type: 'tool', id: 'mcp-app', data: { toolNames: ['search'] } }, + { type: 'tool', id: 'tool-app' } + ]); + }); + + it('extracts agent, dataset, skill and model resources with stable deduplication', () => { + const resources = extractAppResources({ + nodes: [ + createNode({ + flowNodeType: FlowNodeTypeEnum.appModule, + pluginId: 'agent-app' + }), + createNode({ + flowNodeType: FlowNodeTypeEnum.agent, + inputs: [ + createInput(NodeInputKeyEnum.datasetSelectList, [ + { datasetId: 'dataset-1' }, + { datasetId: 'dataset-1' } + ]), + createInput(NodeInputKeyEnum.skills, [{ skillId: 'skill-1' }]), + createInput(NodeInputKeyEnum.aiModelId, 'llm-model-id'), + createInput(NodeInputKeyEnum.datasetSearchUsingReRank, true), + createInput(NodeInputKeyEnum.datasetSearchUsingExtensionQuery, true), + createInput(NodeInputKeyEnum.datasetSearchRerankModelId, 'rerank-model-id'), + createInput(NodeInputKeyEnum.datasetSearchExtensionModelId, 'extension-model-id'), + createInput(NodeInputKeyEnum.datasetParams, { + datasets: [{ datasetId: 'dataset-2' }], + [NodeInputKeyEnum.datasetSearchUsingReRank]: true, + [NodeInputKeyEnum.datasetSearchRerankModelId]: 'nested-rerank-model-id', + [NodeInputKeyEnum.datasetSearchUsingExtensionQuery]: true, + [NodeInputKeyEnum.datasetSearchExtensionModelId]: 'nested-extension-model-id' + }) + ] + }), + createNode({ + inputs: [ + createInput(NodeInputKeyEnum.skills, [{ skillId: 'skill-1' }]), + // 外部节点可以声明同名参数,不能计入 FastGPT 系统模型资源。 + createInput(NodeInputKeyEnum.aiModelId, 'plugin-model-id') + ] + }) + ], + chatConfig: { + questionGuide: { open: true, modelId: 'guide-model-id' }, + ttsConfig: { type: 'model', modelId: 'tts-model-id' } + } as any + }); + + expect(resources).toEqual([ + { type: 'agent', id: 'agent-app' }, + { type: 'dataset', id: 'dataset-1' }, + { type: 'dataset', id: 'dataset-2' }, + { type: 'model', id: 'extension-model-id', data: { modelType: 'llm' } }, + { type: 'model', id: 'guide-model-id', data: { modelType: 'llm' } }, + { type: 'model', id: 'llm-model-id', data: { modelType: 'llm' } }, + { type: 'model', id: 'nested-extension-model-id', data: { modelType: 'llm' } }, + { type: 'model', id: 'nested-rerank-model-id', data: { modelType: 'rerank' } }, + { type: 'model', id: 'rerank-model-id', data: { modelType: 'rerank' } }, + { type: 'model', id: 'tts-model-id', data: { modelType: 'tts' } }, + { type: 'skill', id: 'skill-1' } + ]); + }); + + it('does not record dynamic canonical model references', () => { + const resources = extractAppResources({ + nodes: [ + createNode({ + flowNodeType: FlowNodeTypeEnum.agent, + inputs: [ + createInput(NodeInputKeyEnum.aiModelId, '{{modelId}}'), + createInput(NodeInputKeyEnum.datasetSearchUsingReRank, false), + createInput(NodeInputKeyEnum.datasetSearchRerankModelId, 'disabled-rerank-model-id') + ] + }) + ], + chatConfig: { + questionGuide: { open: true, modelId: '{{guideModelId}}' } + } as any + }); + + expect(resources).toEqual([]); + }); + + it('records selected workflow apps as tool resources', () => { + const resources = extractAppResources({ + nodes: [ + createNode({ + flowNodeType: FlowNodeTypeEnum.agent, + inputs: [ + createInput(NodeInputKeyEnum.selectedTools, [ + { + id: 'personal-workflow-tool', + flowNodeType: FlowNodeTypeEnum.appModule + }, + { id: 'personal-plugin-tool', flowNodeType: FlowNodeTypeEnum.pluginModule } + ]) + ] + }) + ] + }); + + expect(resources).toEqual([ + { type: 'tool', id: 'plugin-tool' }, + { type: 'tool', id: 'workflow-tool' } + ]); + }); +}); + +describe('nodeHasDynamicInput', () => { + it('keeps the dynamic source marker separate from the resolved value', () => { + const node = createNode({ + inputs: [ + createInput(NodeInputKeyEnum.datasetSelectList, [{ datasetId: 'runtime-dataset' }], true) + ] + }); + + expect(nodeHasDynamicInput(undefined, [NodeInputKeyEnum.datasetSelectList])).toBe(false); + expect(nodeHasDynamicInput(node, [NodeInputKeyEnum.datasetSelectList])).toBe(true); + expect(nodeHasDynamicInput(node, [NodeInputKeyEnum.datasetParams])).toBe(false); + }); +}); + +describe('mergeAppResources', () => { + it('keeps model types separate and lets a whole toolset override child tools', () => { + expect( + mergeAppResources([ + { type: 'model', id: 'same-model', data: { modelType: 'rerank' } }, + { type: 'tool', id: 'toolset', data: { toolNames: ['b'] } }, + { type: 'model', id: 'same-model', data: { modelType: 'llm' } }, + { type: 'tool', id: 'toolset' }, + { type: 'tool', id: 'toolset', data: { toolNames: ['a'] } }, + { type: 'tool', id: 'empty-names', data: { toolNames: [] } } + ]) + ).toEqual([ + { type: 'model', id: 'same-model', data: { modelType: 'llm' } }, + { type: 'model', id: 'same-model', data: { modelType: 'rerank' } }, + { type: 'tool', id: 'empty-names' }, + { type: 'tool', id: 'toolset' } + ]); + }); +}); + +describe('resolveStoredAppResources', () => { + it('extracts from nodes and merges legacy skill refs when resources is missing', () => { + expect( + resolveStoredAppResources({ + nodes: [ + createNode({ + flowNodeType: FlowNodeTypeEnum.appModule, + pluginId: 'agent-1' + }) + ], + resourceRefs: { skillIds: ['legacy-skill'] } + }) + ).toEqual([ + { type: 'agent', id: 'agent-1' }, + { type: 'skill', id: 'legacy-skill' } + ]); + }); + + it('keeps an empty array as an empty snapshot', () => { + expect( + resolveStoredAppResources({ + resources: [], + nodes: [ + createNode({ + flowNodeType: FlowNodeTypeEnum.appModule, + pluginId: 'agent-1' + }) + ] + }) + ).toEqual([]); + }); + + it('normalizes legacy child tool ids in a stored snapshot', () => { + expect( + resolveStoredAppResources({ + resources: [{ type: 'tool', id: 'mcp-mcp-app/search' }] + }) + ).toEqual([{ type: 'tool', id: 'mcp-app', data: { toolNames: ['search'] } }]); + }); +}); + +describe('splitExtractedAppResources', () => { + it('does not treat toolNames as a permission delta', () => { + const { kept, added } = splitExtractedAppResources({ + extracted: [{ type: 'tool', id: 'mcp-app', data: { toolNames: ['a', 'b'] } }], + baseline: [{ type: 'tool', id: 'mcp-app', data: { toolNames: ['a'] } }] + }); + expect(added).toEqual([]); + expect(kept).toEqual([{ type: 'tool', id: 'mcp-app', data: { toolNames: ['a', 'b'] } }]); + }); + + it('splits newly added ACL resources from kept baseline resources', () => { + const { kept, added } = splitExtractedAppResources({ + extracted: [ + { type: 'dataset', id: 'old-dataset' }, + { type: 'skill', id: 'new-skill' }, + { type: 'model', id: 'gpt', data: { modelType: 'llm' } } + ], + baseline: [{ type: 'dataset', id: 'old-dataset' }] + }); + expect(kept).toEqual([ + { type: 'dataset', id: 'old-dataset' }, + { type: 'model', id: 'gpt', data: { modelType: 'llm' } } + ]); + expect(added).toEqual([{ type: 'skill', id: 'new-skill' }]); + }); +}); diff --git a/packages/service/test/core/app/tool/httpTool/entity.test.ts b/packages/service/test/core/app/tool/httpTool/entity.test.ts index a3fe42dafa50..e690f755f60a 100644 --- a/packages/service/test/core/app/tool/httpTool/entity.test.ts +++ b/packages/service/test/core/app/tool/httpTool/entity.test.ts @@ -1,8 +1,12 @@ import { describe, it, expect, vi, beforeEach } from 'vitest'; -const { mockMongoAppFind } = vi.hoisted(() => ({ - mockMongoAppFind: vi.fn() -})); +const { mockMongoAppFind, mockMongoAppVersionAggregate, mockMongoAppVersionFind } = vi.hoisted( + () => ({ + mockMongoAppFind: vi.fn(), + mockMongoAppVersionAggregate: vi.fn(), + mockMongoAppVersionFind: vi.fn() + }) +); vi.mock('@fastgpt/service/core/app/schema', () => ({ MongoApp: { @@ -10,69 +14,37 @@ vi.mock('@fastgpt/service/core/app/schema', () => ({ } })); -import { getHttpToolsets } from '@fastgpt/service/core/app/tool/httpTool/entity'; +vi.mock('@fastgpt/service/core/app/version/schema', () => ({ + MongoAppVersion: { + find: mockMongoAppVersionFind, + aggregate: mockMongoAppVersionAggregate + } +})); -const setupFindReturn = (result: any) => { - const leanFn = vi.fn().mockResolvedValue(result); - mockMongoAppFind.mockReturnValue({ lean: leanFn }); - return { leanFn }; -}; +import { getHttpToolsets } from '@fastgpt/service/core/app/tool/httpTool/entity'; beforeEach(() => { vi.clearAllMocks(); + mockMongoAppVersionAggregate.mockResolvedValue([]); + mockMongoAppVersionFind.mockReturnValue({ lean: vi.fn().mockResolvedValue([]) }); }); describe('getHttpToolsets', () => { - it('should query MongoApp with teamId, ids, and field, and return lean result', async () => { - const docs = [ - { _id: 'id1', modules: [{ toolConfig: { httpToolSet: { toolList: [] } } }] }, - { _id: 'id2', modules: [] } - ]; - const { leanFn } = setupFindReturn(docs); - - const field = { _id: true, modules: true }; + it('returns toolset metadata without attaching workflow nodes', async () => { + mockMongoAppFind.mockReturnValue({ + lean: vi.fn().mockResolvedValue([{ _id: 'id1', publishedVersionId: 'version-id' }]) + }); const res = await getHttpToolsets({ teamId: 'team1', - ids: ['id1', 'id2'], - field + ids: ['id1'], + field: { _id: true } }); - expect(mockMongoAppFind).toHaveBeenCalledTimes(1); - expect(mockMongoAppFind).toHaveBeenCalledWith( - { teamId: 'team1', _id: { $in: ['id1', 'id2'] } }, - field - ); - expect(leanFn).toHaveBeenCalledTimes(1); - expect(res).toBe(docs); - }); - - it('should pass undefined field when not provided', async () => { - setupFindReturn([]); - - await getHttpToolsets({ teamId: 'team1', ids: ['id1'] }); - expect(mockMongoAppFind).toHaveBeenCalledWith( { teamId: 'team1', _id: { $in: ['id1'] } }, - undefined + { _id: true, publishedVersionId: true } ); - }); - - it('should handle empty ids array', async () => { - const { leanFn } = setupFindReturn([]); - - const res = await getHttpToolsets({ teamId: 'team1', ids: [] }); - - expect(mockMongoAppFind).toHaveBeenCalledWith({ teamId: 'team1', _id: { $in: [] } }, undefined); - expect(leanFn).toHaveBeenCalledTimes(1); - expect(res).toEqual([]); - }); - - it('should propagate rejection from lean()', async () => { - const leanFn = vi.fn().mockRejectedValue(new Error('db error')); - mockMongoAppFind.mockReturnValue({ lean: leanFn }); - - await expect( - getHttpToolsets({ teamId: 'team1', ids: ['id1'], field: { _id: true } }) - ).rejects.toThrow('db error'); + expect(res).toEqual([{ _id: 'id1', publishedVersionId: 'version-id' }]); + expect(mockMongoAppVersionFind).not.toHaveBeenCalled(); }); }); diff --git a/packages/service/test/core/app/tool/mcpTool/entity.test.ts b/packages/service/test/core/app/tool/mcpTool/entity.test.ts index 8d584d9881c0..74a1454da7ab 100644 --- a/packages/service/test/core/app/tool/mcpTool/entity.test.ts +++ b/packages/service/test/core/app/tool/mcpTool/entity.test.ts @@ -1,8 +1,12 @@ import { describe, it, expect, vi, beforeEach } from 'vitest'; -const { mockMongoAppFind } = vi.hoisted(() => ({ - mockMongoAppFind: vi.fn() -})); +const { mockMongoAppFind, mockMongoAppVersionAggregate, mockMongoAppVersionFind } = vi.hoisted( + () => ({ + mockMongoAppFind: vi.fn(), + mockMongoAppVersionAggregate: vi.fn(), + mockMongoAppVersionFind: vi.fn() + }) +); vi.mock('@fastgpt/service/core/app/schema', () => ({ MongoApp: { @@ -10,69 +14,37 @@ vi.mock('@fastgpt/service/core/app/schema', () => ({ } })); -import { getMcpToolsets } from '@fastgpt/service/core/app/tool/mcpTool/entity'; +vi.mock('@fastgpt/service/core/app/version/schema', () => ({ + MongoAppVersion: { + find: mockMongoAppVersionFind, + aggregate: mockMongoAppVersionAggregate + } +})); -const setupFindReturn = (result: any) => { - const leanFn = vi.fn().mockResolvedValue(result); - mockMongoAppFind.mockReturnValue({ lean: leanFn }); - return { leanFn }; -}; +import { getMcpToolsets } from '@fastgpt/service/core/app/tool/mcpTool/entity'; beforeEach(() => { vi.clearAllMocks(); + mockMongoAppVersionAggregate.mockResolvedValue([]); + mockMongoAppVersionFind.mockReturnValue({ lean: vi.fn().mockResolvedValue([]) }); }); describe('getMcpToolsets', () => { - it('should query MongoApp with teamId, ids, and field, and return lean result', async () => { - const docs = [ - { _id: 'id1', modules: [{ toolConfig: { mcpToolSet: { toolList: [] } } }] }, - { _id: 'id2', modules: [] } - ]; - const { leanFn } = setupFindReturn(docs); - - const field = { _id: true, modules: true }; + it('returns toolset metadata without attaching workflow nodes', async () => { + mockMongoAppFind.mockReturnValue({ + lean: vi.fn().mockResolvedValue([{ _id: 'id1', publishedVersionId: 'version-id' }]) + }); const res = await getMcpToolsets({ teamId: 'team1', - ids: ['id1', 'id2'], - field + ids: ['id1'], + field: { _id: true } }); - expect(mockMongoAppFind).toHaveBeenCalledTimes(1); - expect(mockMongoAppFind).toHaveBeenCalledWith( - { teamId: 'team1', _id: { $in: ['id1', 'id2'] } }, - field - ); - expect(leanFn).toHaveBeenCalledTimes(1); - expect(res).toBe(docs); - }); - - it('should pass undefined field when not provided', async () => { - setupFindReturn([]); - - await getMcpToolsets({ teamId: 'team1', ids: ['id1'] }); - expect(mockMongoAppFind).toHaveBeenCalledWith( { teamId: 'team1', _id: { $in: ['id1'] } }, - undefined + { _id: true, publishedVersionId: true } ); - }); - - it('should handle empty ids array', async () => { - const { leanFn } = setupFindReturn([]); - - const res = await getMcpToolsets({ teamId: 'team1', ids: [] }); - - expect(mockMongoAppFind).toHaveBeenCalledWith({ teamId: 'team1', _id: { $in: [] } }, undefined); - expect(leanFn).toHaveBeenCalledTimes(1); - expect(res).toEqual([]); - }); - - it('should propagate rejection from lean()', async () => { - const leanFn = vi.fn().mockRejectedValue(new Error('db error')); - mockMongoAppFind.mockReturnValue({ lean: leanFn }); - - await expect( - getMcpToolsets({ teamId: 'team1', ids: ['id1'], field: { _id: true } }) - ).rejects.toThrow('db error'); + expect(res).toEqual([{ _id: 'id1', publishedVersionId: 'version-id' }]); + expect(mockMongoAppVersionFind).not.toHaveBeenCalled(); }); }); diff --git a/packages/service/test/core/app/tool/utils/client.test.ts b/packages/service/test/core/app/tool/utils/client.test.ts index 9e3fb82f5718..d91ccfcdba23 100644 --- a/packages/service/test/core/app/tool/utils/client.test.ts +++ b/packages/service/test/core/app/tool/utils/client.test.ts @@ -7,6 +7,7 @@ const mocks = vi.hoisted(() => ({ findById: vi.fn(), find: vi.fn(), getAppVersionById: vi.fn(), + getAppLatestVersion: vi.fn(), getSystemToolDetail: vi.fn() })); @@ -19,6 +20,7 @@ vi.mock('@fastgpt/service/core/app/schema', () => ({ vi.mock('@fastgpt/service/core/app/version/controller', () => ({ getAppVersionById: mocks.getAppVersionById, + getAppLatestVersion: mocks.getAppLatestVersion, checkIsLatestVersion: vi.fn() })); @@ -58,6 +60,19 @@ const getRuntimeSchemaFieldPaths = (value: unknown, path = '$'): string[] => { describe('getClientToolPreviewNode', () => { beforeEach(() => { vi.clearAllMocks(); + const getWorkflowFromApp = (app: any) => ({ + versionId: '507f1f77bcf86cd799439099', + versionName: app?.name, + nodes: app?.modules ?? [], + edges: app?.edges ?? [], + chatConfig: app?.chatConfig + }); + mocks.getAppVersionById.mockImplementation(({ app }: { app?: any }) => + Promise.resolve(getWorkflowFromApp(app)) + ); + mocks.getAppLatestVersion.mockImplementation((appId: string, app?: any) => + Promise.resolve(getWorkflowFromApp(app)) + ); }); it.each(['mcp', 'http'] as const)( @@ -286,150 +301,6 @@ describe('getClientToolPreviewNode', () => { expect(getRuntimeSchemaFieldPaths(result)).toEqual([]); }); - it.each([undefined, {}])( - 'removes only legacy MCP hidden configuration when adding a node (toolConfig: %j)', - async (toolConfig) => { - const appId = '507f1f77bcf86cd799439031'; - const tool = { - name: 'search', - description: 'Search', - inputSchema: { type: 'object', properties: { query: { type: 'string' } } } - }; - const headers = { Authorization: { value: 'legacy-token' } }; - const businessValue = { requestSchema: 'business-data' }; - const app = { - _id: appId, - teamId: '507f1f77bcf86cd799439032', - type: AppTypeEnum.mcpToolSet, - name: 'Legacy', - avatar: 'mcp.svg', - modules: [ - { - flowNodeType: 'toolSet', - toolConfig, - inputs: [ - { - key: NodeInputKeyEnum.toolSetData, - label: 'Old config', - renderTypeList: ['hidden'], - value: { url: 'https://example.com/mcp', headerSecret: headers, toolList: [tool] } - }, - { key: 'options', label: 'Options', renderTypeList: ['input'], value: businessValue }, - { - key: NodeInputKeyEnum.toolSetData, - label: 'User field', - renderTypeList: ['input'], - value: 'ordinary-value' - } - ], - outputs: [] - } - ] - }; - const original = structuredClone(app); - mocks.findById.mockReturnValueOnce({ lean: async () => app }); - mocks.find.mockReturnValueOnce({ - lean: async () => [ - { - name: tool.name, - modules: [ - { - inputs: [ - { value: { ...tool, url: 'https://example.com/mcp', headerSecret: headers } } - ] - } - ] - } - ] - }); - const preview = await getClientToolPreviewNode({ appId, versionId: '' }); - expect(preview.toolConfig).toEqual({ - mcpToolSet: { toolId: appId, toolList: [{ name: 'search', description: 'Search' }] } - }); - expect(preview.inputs).toHaveLength(2); - expect(preview.inputs[0]).toMatchObject({ key: 'options', value: businessValue }); - expect(preview.inputs[1]).toMatchObject({ - key: NodeInputKeyEnum.toolSetData, - value: 'ordinary-value' - }); - expect(JSON.stringify(preview)).not.toContain('inputSchema'); - expect(app).toEqual(original); - } - ); - - it.each([ - { toolList: [] }, - { toolList: [{ name: 'search', description: 'Search', inputSchema: { type: 'object' } }] } - ])('adds an inline MCP toolset with an empty legacy id: %j', async ({ toolList }) => { - const appId = '507f1f77bcf86cd799439031'; - const app = { - _id: appId, - teamId: '507f1f77bcf86cd799439032', - type: AppTypeEnum.mcpToolSet, - name: 'Legacy', - avatar: 'mcp.svg', - modules: [ - { - flowNodeType: 'toolSet', - toolConfig: { mcpToolSet: { toolId: '', url: 'https://example.com/mcp', toolList } }, - inputs: [], - outputs: [] - } - ] - }; - const original = structuredClone(app); - mocks.findById.mockReturnValueOnce({ lean: async () => app }); - const preview = await getClientToolPreviewNode({ appId, versionId: '' }); - expect(preview.toolConfig?.mcpToolSet).toMatchObject({ toolId: appId }); - expect(getRuntimeSchemaFieldPaths(preview)).toEqual([]); - expect(app).toEqual(original); - }); - - it('hydrates legacy MCP toolset data under toolConfig', async () => { - const appId = '507f1f77bcf86cd799439031'; - mocks.findById.mockReturnValueOnce({ - lean: vi.fn().mockResolvedValue({ - _id: appId, - teamId: '507f1f77bcf86cd799439032', - type: AppTypeEnum.mcpToolSet, - name: 'Legacy MCP Tools', - avatar: 'mcp.svg', - intro: 'Legacy MCP toolset', - modules: [{ flowNodeType: 'toolSet', inputs: [] }] - }) - }); - mocks.find.mockReturnValueOnce({ - lean: vi.fn().mockResolvedValue([ - { - name: 'search', - modules: [ - { - inputs: [ - { - value: { - name: 'search', - description: 'Search tool', - inputSchema: { type: 'object' }, - url: 'https://mcp.example.com' - } - } - ] - } - ] - } - ]) - }); - - const result = await getClientToolPreviewNode({ appId, lang: 'en' }); - - expect(result.toolConfig?.mcpToolSet).toMatchObject({ - toolId: appId, - toolList: [{ name: 'search', description: 'Search tool' }] - }); - expect(JSON.stringify(result.toolConfig)).not.toContain('inputSchema'); - expect(getRuntimeSchemaFieldPaths(result)).toEqual([]); - }); - it('applies defaultToAgentGenerated over a workflow plugin input selection', async () => { const appId = '507f1f77bcf86cd799439011'; mocks.findById.mockReturnValueOnce({ diff --git a/packages/service/test/core/app/version/controller.test.ts b/packages/service/test/core/app/version/controller.test.ts index fa8e43578fa8..414bfcd9deb8 100644 --- a/packages/service/test/core/app/version/controller.test.ts +++ b/packages/service/test/core/app/version/controller.test.ts @@ -1,28 +1,66 @@ import { beforeEach, describe, expect, it, vi } from 'vitest'; -import { FlowNodeInputTypeEnum } from '@fastgpt/global/core/workflow/node/constant'; +import { + FlowNodeInputTypeEnum, + FlowNodeTypeEnum +} from '@fastgpt/global/core/workflow/node/constant'; import { NodeInputKeyEnum } from '@fastgpt/global/core/workflow/constants'; -const { findOneMock, findAppByIdMock } = vi.hoisted(() => ({ - findOneMock: vi.fn(), - findAppByIdMock: vi.fn() -})); +const { findOneMock, findMock, aggregateMock, findAppByIdMock, updateVersionMock, updateAppMock } = + vi.hoisted(() => ({ + findOneMock: vi.fn(), + findMock: vi.fn(), + aggregateMock: vi.fn(), + findAppByIdMock: vi.fn(), + updateVersionMock: vi.fn(), + updateAppMock: vi.fn() + })); vi.mock('@fastgpt/service/core/app/version/schema', () => ({ MongoAppVersion: { - findOne: findOneMock + findOne: findOneMock, + find: findMock, + aggregate: aggregateMock, + updateOne: updateVersionMock } })); vi.mock('@fastgpt/service/core/app/schema', () => ({ MongoApp: { - findById: findAppByIdMock + findById: findAppByIdMock, + updateOne: updateAppMock } })); +vi.mock('@fastgpt/service/common/mongo/sessionRun', async (importOriginal) => { + return importOriginal(); +}); + import { + getAppDraftVersion, getAppLatestVersion, - getAppVersionById + getAppVersionById, + getAppPublishedWorkflowMap, + updateAppPublishedVersion } from '@fastgpt/service/core/app/version/controller'; +import type { ClientSession } from '@fastgpt/service/common/mongo'; +import { MongoTransactionConflictError } from '@fastgpt/service/common/mongo/sessionRun'; + +const createAgentVersion = (resources?: unknown) => ({ + _id: '507f1f77bcf86cd799439011', + nodes: [ + { + nodeId: 'agent-node', + name: 'Agent', + flowNodeType: FlowNodeTypeEnum.appModule, + pluginId: 'legacy-agent-id', + inputs: [], + outputs: [] + } + ], + edges: [], + chatConfig: undefined, + ...(resources === undefined ? {} : { resources }) +}); describe('getAppLatestVersion', () => { beforeEach(() => { @@ -78,7 +116,8 @@ describe('getAppLatestVersion', () => { targetHandle: 'start-target-left' } ], - chatConfig: {} + chatConfig: {}, + resources: [] }; const leanMock = vi.fn().mockResolvedValue(version); findOneMock.mockReturnValue({ @@ -94,35 +133,17 @@ describe('getAppLatestVersion', () => { expect(result.nodes[0].inputs[0].inputList?.[0]).not.toHaveProperty('value'); }); - it('normalizes the app fallback when no published version exists', async () => { + it('returns an empty workflow when no published version exists', async () => { findOneMock.mockReturnValue({ sort: vi.fn().mockReturnValue({ lean: vi.fn().mockResolvedValue(undefined) }) }); const result = await getAppLatestVersion('app-id', { - name: 'Legacy app', - modules: [ - { - nodeId: 'userGuide', - name: 'System config', - flowNodeType: 'userGuide', - inputs: [ - { - key: NodeInputKeyEnum.welcomeText, - label: 'Welcome text', - value: 'Legacy welcome', - renderTypeList: [FlowNodeInputTypeEnum.hidden] - } - ], - outputs: [] - } - ], - edges: [], - chatConfig: {} + name: 'App without version' } as any); expect(result.nodes).toEqual([]); - expect(result.chatConfig.welcomeConfig?.welcomeText).toBe('Legacy welcome'); + expect(result.resources).toEqual([]); }); it('preserves a legacy version config when the current app config differs', async () => { @@ -146,7 +167,8 @@ describe('getAppLatestVersion', () => { } ], edges: [], - chatConfig: undefined + chatConfig: undefined, + resources: [] }; findOneMock.mockReturnValue({ sort: vi.fn().mockReturnValue({ lean: vi.fn().mockResolvedValue(version) }) @@ -165,6 +187,68 @@ describe('getAppLatestVersion', () => { expect(result.chatConfig.welcomeConfig?.welcomeText).toBe('Legacy welcome'); expect(result.chatConfig.welcomeText).toBe('Legacy welcome'); }); + + it('reads the version pointed by publishedVersionId', async () => { + const version = { + _id: '507f1f77bcf86cd799439011', + versionName: 'Pinned published', + nodes: [], + edges: [], + chatConfig: {}, + resources: [{ type: 'skill', id: 'pinned-skill' }] + }; + findOneMock.mockReturnValue({ + lean: vi.fn().mockResolvedValue(version) + }); + + const result = await getAppLatestVersion('app-id', { + publishedVersionId: '507f1f77bcf86cd799439011' + } as any); + + expect(findOneMock).toHaveBeenCalledWith({ + _id: '507f1f77bcf86cd799439011', + appId: 'app-id' + }); + expect(result.resources).toEqual([{ type: 'skill', id: 'pinned-skill' }]); + }); +}); + +describe('getAppDraftVersion', () => { + beforeEach(() => { + vi.clearAllMocks(); + }); + + it('always reads the latest Version by time instead of an App pointer', async () => { + const version = createAgentVersion([]); + const sortMock = vi.fn().mockReturnValue({ + lean: vi.fn().mockResolvedValue(version) + }); + findOneMock.mockReturnValue({ sort: sortMock }); + + const result = await getAppDraftVersion('app-id'); + + expect(findOneMock).toHaveBeenCalledWith({ appId: 'app-id' }); + expect(sortMock).toHaveBeenCalledWith({ time: -1, _id: -1 }); + expect(findAppByIdMock).not.toHaveBeenCalled(); + expect(result).toBe(version); + }); + + it('uses the provided transaction session when reading the latest Version', async () => { + const version = createAgentVersion([]); + const session = {} as ClientSession; + const sessionMock = vi.fn(); + const query = { + session: sessionMock, + lean: vi.fn().mockResolvedValue(version) + }; + const sortMock = vi.fn().mockReturnValue(query); + findOneMock.mockReturnValue({ sort: sortMock }); + + const result = await getAppDraftVersion('app-id', session); + + expect(sessionMock).toHaveBeenCalledWith(session); + expect(result).toBe(version); + }); }); describe('getAppVersionById', () => { @@ -173,42 +257,242 @@ describe('getAppVersionById', () => { findAppByIdMock.mockReturnValue({ lean: vi.fn().mockResolvedValue(undefined) }); }); - it('preserves a legacy version config when the current app config differs', async () => { - const version = { - _id: '507f1f77bcf86cd799439011', - versionName: 'Legacy version', - nodes: [ - { - nodeId: 'userGuide', - name: 'System config', - flowNodeType: 'userGuide', - inputs: [ - { - key: NodeInputKeyEnum.instruction, - label: 'Instruction', - value: 'Legacy instruction', - renderTypeList: [FlowNodeInputTypeEnum.hidden] - } - ], - outputs: [] - } - ], - edges: [], - chatConfig: undefined - }; - findOneMock.mockReturnValue({ lean: vi.fn().mockResolvedValue(version) }); + it('extracts resources from a version that has not been migrated', async () => { + findOneMock.mockReturnValue({ + lean: vi.fn().mockResolvedValue(createAgentVersion()) + }); const result = await getAppVersionById({ appId: 'app-id', - versionId: '507f1f77bcf86cd799439011', - app: { - chatConfig: { - instruction: 'Current instruction' + versionId: '507f1f77bcf86cd799439011' + }); + + expect(result.resources).toEqual([{ type: 'agent', id: 'legacy-agent-id' }]); + }); +}); + +describe('getAppPublishedWorkflowMap', () => { + beforeEach(() => { + vi.clearAllMocks(); + }); + + it('uses the pointer version when it belongs to the same app', async () => { + const versionId = '507f1f77bcf86cd799439011'; + findMock.mockReturnValue({ + lean: vi.fn().mockResolvedValue([ + { + _id: versionId, + appId: 'app-id', + nodes: [{ nodeId: 'pointer-node' }] } + ]) + }); + + const result = await getAppPublishedWorkflowMap([ + { + _id: 'app-id', + publishedVersionId: versionId } as any + ]); + + expect(aggregateMock).not.toHaveBeenCalled(); + expect(result.get('app-id')?.nodes.map((node) => node.nodeId)).toEqual(['pointer-node']); + }); + + it('ignores a pointer that belongs to another app and falls back to latest publish', async () => { + const foreignVersionId = '507f1f77bcf86cd799439012'; + findMock.mockReturnValue({ + lean: vi.fn().mockResolvedValue([ + { + _id: foreignVersionId, + appId: 'other-app', + nodes: [{ nodeId: 'foreign-node' }] + } + ]) }); + aggregateMock.mockResolvedValue([ + { + _id: 'app-id', + doc: { + _id: '507f1f77bcf86cd799439011', + appId: 'app-id', + nodes: [{ nodeId: 'own-node' }] + } + } + ]); - expect(result.nodes).toEqual([]); - expect(result.chatConfig.instruction).toBe('Legacy instruction'); + const result = await getAppPublishedWorkflowMap([ + { + _id: 'app-id', + publishedVersionId: foreignVersionId + } as any + ]); + + expect(aggregateMock).toHaveBeenCalled(); + expect(result.get('app-id')?.nodes.map((node) => node.nodeId)).toEqual(['own-node']); + }); +}); + +describe('updateAppPublishedVersion', () => { + beforeEach(() => { + vi.clearAllMocks(); + }); + + it('updates the pointed version and guards the App pointer with CAS', async () => { + const session = {} as ClientSession; + const appQuery = { + session: vi.fn().mockReturnThis(), + lean: vi.fn().mockResolvedValue({ + _id: 'app-id', + publishedVersionId: '507f1f77bcf86cd799439011' + }) + }; + const versionQuery = { + session: vi.fn().mockReturnThis(), + lean: vi.fn().mockResolvedValue({ _id: '507f1f77bcf86cd799439011' }) + }; + findAppByIdMock.mockReturnValue(appQuery); + findOneMock.mockReturnValue(versionQuery); + updateVersionMock.mockResolvedValue({ matchedCount: 1 }); + updateAppMock.mockResolvedValue({ matchedCount: 1 }); + + await updateAppPublishedVersion({ + appId: 'app-id', + nodes: [], + resources: [], + session + }); + + expect(appQuery.session).toHaveBeenCalledWith(session); + expect(versionQuery.session).toHaveBeenCalledWith(session); + expect(updateVersionMock).toHaveBeenCalledWith( + { _id: '507f1f77bcf86cd799439011', appId: 'app-id' }, + { $set: { nodes: [], resources: [] } }, + { session } + ); + expect(updateAppMock).toHaveBeenCalledWith( + { _id: 'app-id', publishedVersionId: '507f1f77bcf86cd799439011' }, + { + $set: expect.objectContaining({ + updateTime: expect.any(Date) + }) + }, + { session } + ); + }); + + it('falls back to the latest published version and repairs a missing pointer', async () => { + const session = {} as ClientSession; + const appQuery = { + session: vi.fn().mockReturnThis(), + lean: vi.fn().mockResolvedValue({ _id: 'app-id' }) + }; + const sortMock = vi.fn(); + const versionQuery = { + sort: sortMock, + session: vi.fn().mockReturnThis(), + lean: vi.fn().mockResolvedValue({ _id: 'published-version' }) + }; + sortMock.mockReturnValue(versionQuery); + findAppByIdMock.mockReturnValue(appQuery); + findOneMock.mockReturnValue(versionQuery); + updateVersionMock.mockResolvedValue({ matchedCount: 1 }); + updateAppMock.mockResolvedValue({ matchedCount: 1 }); + + await updateAppPublishedVersion({ + appId: 'app-id', + nodes: [], + resources: [], + session + }); + + expect(findOneMock).toHaveBeenCalledWith({ appId: 'app-id', isPublish: true }, '_id'); + expect(sortMock).toHaveBeenCalledWith({ time: -1, _id: -1 }); + expect(updateAppMock).toHaveBeenCalledWith( + { + _id: 'app-id', + $or: [{ publishedVersionId: null }, { publishedVersionId: { $exists: false } }] + }, + { + $set: expect.objectContaining({ + publishedVersionId: 'published-version', + updateTime: expect.any(Date) + }) + }, + { session } + ); + }); + + it('repairs a pointer whose version no longer belongs to the App', async () => { + const session = {} as ClientSession; + const appQuery = { + session: vi.fn().mockReturnThis(), + lean: vi.fn().mockResolvedValue({ + _id: 'app-id', + publishedVersionId: '507f1f77bcf86cd799439011' + }) + }; + const pointerQuery = { + session: vi.fn().mockReturnThis(), + lean: vi.fn().mockResolvedValue(undefined) + }; + const sortMock = vi.fn(); + const fallbackQuery = { + sort: sortMock, + session: vi.fn().mockReturnThis(), + lean: vi.fn().mockResolvedValue({ _id: 'published-version' }) + }; + sortMock.mockReturnValue(fallbackQuery); + findAppByIdMock.mockReturnValue(appQuery); + findOneMock.mockReturnValueOnce(pointerQuery).mockReturnValueOnce(fallbackQuery); + updateVersionMock.mockResolvedValue({ matchedCount: 1 }); + updateAppMock.mockResolvedValue({ matchedCount: 1 }); + + await updateAppPublishedVersion({ + appId: 'app-id', + nodes: [], + resources: [], + session + }); + + expect(updateAppMock).toHaveBeenCalledWith( + { + _id: 'app-id', + publishedVersionId: '507f1f77bcf86cd799439011' + }, + { + $set: expect.objectContaining({ + publishedVersionId: 'published-version', + updateTime: expect.any(Date) + }) + }, + { session } + ); + }); + + it('raises a transaction conflict when the App pointer changed', async () => { + const session = {} as ClientSession; + findAppByIdMock.mockReturnValue({ + session: vi.fn().mockReturnThis(), + lean: vi.fn().mockResolvedValue({ + _id: 'app-id', + publishedVersionId: '507f1f77bcf86cd799439011' + }) + }); + findOneMock.mockReturnValue({ + session: vi.fn().mockReturnThis(), + lean: vi.fn().mockResolvedValue({ _id: '507f1f77bcf86cd799439011' }) + }); + updateVersionMock.mockResolvedValue({ matchedCount: 1 }); + updateAppMock.mockResolvedValue({ matchedCount: 0 }); + + await expect( + updateAppPublishedVersion({ + appId: 'app-id', + nodes: [], + resources: [], + session + }) + ).rejects.toBeInstanceOf(MongoTransactionConflictError); }); }); diff --git a/packages/service/test/core/workflow/dispatch/abandoned/runApp.test.ts b/packages/service/test/core/workflow/dispatch/abandoned/runApp.test.ts index dc6b202bb381..52b94db3c8bc 100644 --- a/packages/service/test/core/workflow/dispatch/abandoned/runApp.test.ts +++ b/packages/service/test/core/workflow/dispatch/abandoned/runApp.test.ts @@ -8,6 +8,7 @@ import { summarizeRuntimeNodeResponses } from '../../../../../core/workflow/disp const runWorkflowMock = vi.fn(); const authAppByTmbIdMock = vi.fn(); const getUserChatInfoMock = vi.fn(); +const getAppVersionByIdMock = vi.fn(); vi.mock('../../../../../core/workflow/dispatch/index', () => ({ runWorkflow: (args: any) => runWorkflowMock(args) @@ -21,6 +22,11 @@ vi.mock('../../../../../support/user/team/utils', () => ({ getUserChatInfo: (...args: any[]) => getUserChatInfoMock(...args) })); +vi.mock('../../../../../core/app/version/controller', () => ({ + getAppVersionById: (...args: any[]) => getAppVersionByIdMock(...args), + getAppPublishedWorkflowMap: vi.fn(async () => new Map()) +})); + import { dispatchAppRequest } from '../../../../../core/workflow/dispatch/abandoned/runApp'; const createParentVariableState = () => @@ -60,16 +66,7 @@ describe('abandoned dispatchAppRequest', () => { teamId: 'team', tmbId: 'child-tmb', modules: [], - edges: [], - chatConfig: { - variables: [ - { - key: 'shared', - type: VariableInputEnum.input, - valueType: WorkflowIOValueTypeEnum.string - } - ] - } + edges: [] } }); getUserChatInfoMock.mockResolvedValue({ @@ -77,6 +74,22 @@ describe('abandoned dispatchAppRequest', () => { externalWorkflowVariables: {} } }); + getAppVersionByIdMock.mockResolvedValue({ + versionId: '', + versionName: 'child', + nodes: [], + edges: [], + chatConfig: { + variables: [ + { + key: 'shared', + type: VariableInputEnum.input, + valueType: WorkflowIOValueTypeEnum.string + } + ] + }, + resources: [] + }); runWorkflowMock.mockImplementation(async (args: any) => { childInitialValue = args.variableState.get('shared'); await args.variableState.set('shared', 'child-value'); @@ -109,6 +122,10 @@ describe('abandoned dispatchAppRequest', () => { tmbId: 'parent-tmb', name: 'parent' }, + runningUserInfo: { + tmbId: 'parent-tmb', + teamId: 'team' + }, workflowStreamResponse: vi.fn(), histories: [], query: [], diff --git a/packages/service/test/core/workflow/dispatch/ai/agent/index.test.ts b/packages/service/test/core/workflow/dispatch/ai/agent/index.test.ts index 8862e89dcc2b..997c7e13e1e9 100644 --- a/packages/service/test/core/workflow/dispatch/ai/agent/index.test.ts +++ b/packages/service/test/core/workflow/dispatch/ai/agent/index.test.ts @@ -560,7 +560,7 @@ describe('dispatchRunAgent user context', () => { expect(getSandboxClientMock).toHaveBeenCalledTimes(1); }); - it('authenticates skill injection with the app owner tmbId for published apps', async () => { + it('uses the runtime member for skill injection without an App resource context', async () => { const { dispatchRunAgent } = await import('@fastgpt/service/core/workflow/dispatch/ai/agent'); const props = createProps(); props.params.useAgentSandbox = true; @@ -583,7 +583,7 @@ describe('dispatchRunAgent user context', () => { expect(injectAgentSkillFilesToSandboxMock).toHaveBeenCalledWith( expect.objectContaining({ teamId: 'team_1', - tmbId: 'app_owner_tmb', + tmbId: 'external_user_tmb', skillIds: ['skill_1'] }) ); diff --git a/packages/service/test/core/workflow/dispatch/ai/agent/sub/app.test.ts b/packages/service/test/core/workflow/dispatch/ai/agent/sub/app.test.ts index 4e4eda21a89c..ee82bdb950ff 100644 --- a/packages/service/test/core/workflow/dispatch/ai/agent/sub/app.test.ts +++ b/packages/service/test/core/workflow/dispatch/ai/agent/sub/app.test.ts @@ -30,7 +30,8 @@ vi.mock('@fastgpt/service/support/permission/app/auth', () => ({ })); vi.mock('@fastgpt/service/core/app/version/controller', () => ({ - getAppVersionById: mocks.getAppVersionById + getAppVersionById: mocks.getAppVersionById, + getAppPublishedWorkflowMap: vi.fn(async () => new Map()) })); vi.mock('@fastgpt/service/support/user/team/utils', () => ({ @@ -150,6 +151,7 @@ describe('agent sub app dispatchPlugin', () => { } }); mocks.getAppVersionById.mockResolvedValue({ + resources: [], nodes: [ { nodeId: 'pluginInput', @@ -409,6 +411,7 @@ describe('agent sub app dispatchPlugin', () => { } }); mocks.getAppVersionById.mockResolvedValue({ + resources: [], nodes: [ { nodeId: 'pluginInput', @@ -510,6 +513,7 @@ describe('agent sub app dispatchPlugin', () => { } }); mocks.getAppVersionById.mockResolvedValue({ + resources: [], nodes: [], edges: [], chatConfig: { variables: [] } @@ -580,6 +584,7 @@ describe('agent sub app dispatchApp', () => { } }); mocks.getAppVersionById.mockResolvedValue({ + resources: [], nodes: [], edges: [], chatConfig: { @@ -677,6 +682,7 @@ describe('agent sub app dispatchApp', () => { it('does not allow workflow tool arguments to override internal variables', async () => { mocks.getAppVersionById.mockResolvedValue({ + resources: [], nodes: [], edges: [], chatConfig: { diff --git a/packages/service/test/core/workflow/dispatch/ai/agent/sub/dataset.test.ts b/packages/service/test/core/workflow/dispatch/ai/agent/sub/dataset.test.ts index cd159e260cc9..d8247618f740 100644 --- a/packages/service/test/core/workflow/dispatch/ai/agent/sub/dataset.test.ts +++ b/packages/service/test/core/workflow/dispatch/ai/agent/sub/dataset.test.ts @@ -7,14 +7,14 @@ const { createLLMResponseMock, defaultSearchDatasetDataMock, filterDatasetsByTmbIdMock, - findDatasetByIdMock, + loadWorkflowDatasetResourceMock, formatModelChars2PointsMock } = vi.hoisted(() => ({ countPromptTokensMock: vi.fn(), createLLMResponseMock: vi.fn(), defaultSearchDatasetDataMock: vi.fn(), filterDatasetsByTmbIdMock: vi.fn(), - findDatasetByIdMock: vi.fn(), + loadWorkflowDatasetResourceMock: vi.fn(), formatModelChars2PointsMock: vi.fn() })); @@ -22,17 +22,14 @@ vi.mock('@fastgpt/service/core/dataset/search', () => ({ defaultSearchDatasetData: defaultSearchDatasetDataMock })); -vi.mock('@fastgpt/service/core/dataset/schema', async (importOriginal) => ({ - ...(await importOriginal()), - MongoDataset: { - findById: findDatasetByIdMock - } -})); - vi.mock('@fastgpt/service/core/dataset/utils', () => ({ filterDatasetsByTmbId: filterDatasetsByTmbIdMock })); +vi.mock('@fastgpt/service/core/workflow/utils/resource', () => ({ + loadWorkflowDatasetResource: loadWorkflowDatasetResourceMock +})); + vi.mock('@fastgpt/service/core/ai/model', () => ({ getEmbeddingModelData: vi.fn(() => ({ model: 'embedding-model', @@ -94,11 +91,9 @@ const llmModelData = (model: string) => describe('dispatchAgentDatasetSearch', () => { beforeEach(() => { vi.clearAllMocks(); - findDatasetByIdMock.mockReturnValue({ - lean: vi.fn().mockResolvedValue({ - vectorModel: 'embedding-model', - vlmModel: 'vlm-model' - }) + loadWorkflowDatasetResourceMock.mockResolvedValue({ + vectorModel: 'embedding-model', + vlmModel: 'vlm-model' }); filterDatasetsByTmbIdMock.mockImplementation(async ({ datasetIds }) => datasetIds); countPromptTokensMock.mockResolvedValue(100); @@ -417,7 +412,7 @@ describe('dispatchAgentDatasetSearch', () => { expect(result).toEqual({ response: 'No authorized dataset selected' }); - expect(findDatasetByIdMock).not.toHaveBeenCalled(); + expect(loadWorkflowDatasetResourceMock).not.toHaveBeenCalled(); expect(defaultSearchDatasetDataMock).not.toHaveBeenCalled(); }); diff --git a/packages/service/test/core/workflow/dispatch/ai/agent/sub/sandbox/prepare.test.ts b/packages/service/test/core/workflow/dispatch/ai/agent/sub/sandbox/prepare.test.ts index 59a5206926e3..01904f431b91 100644 --- a/packages/service/test/core/workflow/dispatch/ai/agent/sub/sandbox/prepare.test.ts +++ b/packages/service/test/core/workflow/dispatch/ai/agent/sub/sandbox/prepare.test.ts @@ -132,6 +132,7 @@ describe('ensureAgentSandboxRuntime', () => { teamId: 'team_1', tmbId: 'tmb_1', skillIds: ['skill_1'], + dynamic: false, workDirectory: '/workspace' }); expect(prepareSandboxRuntimeMirrorsMock).toHaveBeenCalledWith({ diff --git a/packages/service/test/core/workflow/dispatch/ai/agent/sub/tool/utils.test.ts b/packages/service/test/core/workflow/dispatch/ai/agent/sub/tool/utils.test.ts index 708b60c19c04..42b50a3fb9ff 100644 --- a/packages/service/test/core/workflow/dispatch/ai/agent/sub/tool/utils.test.ts +++ b/packages/service/test/core/workflow/dispatch/ai/agent/sub/tool/utils.test.ts @@ -11,6 +11,7 @@ import { WorkflowIOValueTypeEnum } from '@fastgpt/global/core/workflow/constants'; import { getAgentRuntimeTools } from '@fastgpt/service/core/workflow/dispatch/ai/agent/sub/tool/utils'; +import { runWithContext } from '@fastgpt/service/core/workflow/utils/context'; import type { NodeToolConfigType } from '@fastgpt/global/core/workflow/type/node'; const { @@ -62,18 +63,21 @@ vi.mock('@fastgpt/service/core/app/tool/systemTool/systemTool.repo', () => ({ } })); -vi.mock('@fastgpt/service/common/logger', () => ({ - LogCategories: { - MODULE: { - AI: { - AGENT: 'agent' - } +vi.mock('@fastgpt/service/common/logger', () => { + const logCategory = new Proxy( + {}, + { + get: () => logCategory } - }, - getLogger: vi.fn(() => ({ - warn: vi.fn() - })) -})); + ); + + return { + LogCategories: logCategory, + getLogger: vi.fn(() => ({ + warn: vi.fn() + })) + }; +}); const mcpInputSchema = { type: 'object', @@ -811,7 +815,7 @@ describe('getAgentRuntimeTools schema loading', () => { tools: [{ id: 'mcp-stripped_mcp_app/search', config: {} }] }); - expect(getMCPChildrenMock).toHaveBeenCalledWith(appMap.stripped_mcp_app); + expect(getMCPChildrenMock).toHaveBeenCalledWith(appMap.stripped_mcp_app, undefined); expect(tools).toHaveLength(1); expect(tools[0].requestSchema.function.parameters).toEqual(getModelToolSchema(mcpInputSchema)); }); @@ -830,7 +834,7 @@ describe('getAgentRuntimeTools schema loading', () => { tools: [{ id: 'legacy_mcp_app', config: {} }] }); - expect(getMCPChildrenMock).toHaveBeenCalledWith(appMap.legacy_mcp_app); + expect(getMCPChildrenMock).toHaveBeenCalledWith(appMap.legacy_mcp_app, undefined); expect(tools).toHaveLength(1); expect(tools[0].requestSchema.function.name).toBe('legacy_mcp_app0'); expect(tools[0].requestSchema.function.parameters).toEqual(getModelToolSchema(mcpInputSchema)); @@ -851,7 +855,7 @@ describe('getAgentRuntimeTools schema loading', () => { tools: [{ id: 'mcp-legacy_mcp_app/search', config: {} }] }); - expect(getMCPChildrenMock).toHaveBeenCalledWith(appMap.legacy_mcp_app); + expect(getMCPChildrenMock).toHaveBeenCalledWith(appMap.legacy_mcp_app, undefined); expect(tools).toHaveLength(1); expect(tools[0].id).toBe('legacy_mcp_appsearch'); expect(tools[0].name).toBe('search'); @@ -962,6 +966,36 @@ describe('getAgentRuntimeTools schema loading', () => { expect(tools[0].requestSchema.function.parameters.required).toEqual(['generated']); }); + it('loads a dynamic MCP tool without using the parent resource snapshot', async () => { + const tools = await runWithContext( + { + mcpClientMemory: {}, + resourceContext: { + teamId: 'team_1', + resources: [], + resourceMap: new Map(), + appMap: new Map(), + workflowMap: new Map(), + datasetMap: new Map(), + skillMap: new Map() + } + }, + () => + getAgentRuntimeTools({ + tmbId: 'tmb_1', + dynamic: true, + tools: [{ id: 'mcp-mcp_app/search', config: {} }] + }) + ); + + expect(tools).toHaveLength(1); + expect(tools[0].name).toBe('search'); + expect(tools[0].dynamic).toBe(true); + expect(authAppByTmbIdMock).toHaveBeenCalledWith( + expect.objectContaining({ appId: 'mcp_app', tmbId: 'tmb_1' }) + ); + }); + it('loads a legacy HTTP tool without isToolParam as an agent tool', async () => { const tools = await getAgentRuntimeTools({ tmbId: 'tmb_1', diff --git a/packages/service/test/core/workflow/dispatch/dataset/search.test.ts b/packages/service/test/core/workflow/dispatch/dataset/search.test.ts index 55b8c2e115f0..05807192636b 100644 --- a/packages/service/test/core/workflow/dispatch/dataset/search.test.ts +++ b/packages/service/test/core/workflow/dispatch/dataset/search.test.ts @@ -6,13 +6,13 @@ import { FlowNodeTypeEnum } from '@fastgpt/global/core/workflow/node/constant'; const { defaultSearchDatasetDataMock, deepRagSearchMock, - findDatasetByIdMock, + loadWorkflowDatasetResourceMock, formatModelChars2PointsMock, usagePushMock } = vi.hoisted(() => ({ defaultSearchDatasetDataMock: vi.fn(), deepRagSearchMock: vi.fn(), - findDatasetByIdMock: vi.fn(), + loadWorkflowDatasetResourceMock: vi.fn(), formatModelChars2PointsMock: vi.fn(), usagePushMock: vi.fn() })); @@ -22,17 +22,14 @@ vi.mock('@fastgpt/service/core/dataset/search', () => ({ deepRagSearch: deepRagSearchMock })); -vi.mock('@fastgpt/service/core/dataset/schema', () => ({ - DatasetCollectionName: 'datasets', - MongoDataset: { - findById: findDatasetByIdMock - } -})); - vi.mock('@fastgpt/service/core/dataset/utils', () => ({ filterDatasetsByTmbId: vi.fn() })); +vi.mock('@fastgpt/service/core/workflow/utils/resource', () => ({ + loadWorkflowDatasetResource: loadWorkflowDatasetResourceMock +})); + vi.mock('@fastgpt/service/core/ai/model', () => ({ getEmbeddingModelData: vi.fn(() => ({ modelId: '68ad85a7463006c963799a01', @@ -78,11 +75,9 @@ import { dispatchDatasetSearch } from '../../../../../core/workflow/dispatch/dat describe('dispatchDatasetSearch', () => { beforeEach(() => { vi.clearAllMocks(); - findDatasetByIdMock.mockReturnValue({ - lean: vi.fn().mockResolvedValue({ - vectorModel: 'embedding-model', - vlmModel: 'gpt-vision' - }) + loadWorkflowDatasetResourceMock.mockResolvedValue({ + vectorModel: 'embedding-model', + vlmModel: 'gpt-vision' }); formatModelChars2PointsMock.mockImplementation( ({ diff --git a/packages/service/test/core/workflow/dispatch/tools/http468.test.ts b/packages/service/test/core/workflow/dispatch/tools/http468.test.ts index 96d34139c7ca..e8d6a5a61d43 100644 --- a/packages/service/test/core/workflow/dispatch/tools/http468.test.ts +++ b/packages/service/test/core/workflow/dispatch/tools/http468.test.ts @@ -5,9 +5,13 @@ import { ContentTypes, NodeInputKeyEnum } from '@fastgpt/global/core/workflow/co const axiosMock = vi.hoisted(() => vi.fn()); const isInternalAddressMock = vi.hoisted(() => vi.fn()); -vi.mock('@fastgpt/service/common/api/axios', () => ({ - axios: axiosMock -})); +vi.mock('@fastgpt/service/common/api/axios', async (importOriginal) => { + const mod = await importOriginal(); + return { + ...mod, + axios: axiosMock + }; +}); vi.mock('@fastgpt/service/common/system/utils', async (importOriginal) => { const mod = await importOriginal(); diff --git a/packages/service/test/core/workflow/dispatch/tools/runTool.test.ts b/packages/service/test/core/workflow/dispatch/tools/runTool.test.ts index 6888e5a49fc2..fb0112d16644 100644 --- a/packages/service/test/core/workflow/dispatch/tools/runTool.test.ts +++ b/packages/service/test/core/workflow/dispatch/tools/runTool.test.ts @@ -52,21 +52,15 @@ vi.mock('@fastgpt/service/core/app/mcp', () => ({ }) })); -vi.mock('@fastgpt/service/common/logger', () => ({ - LogCategories: { - MODULE: { - APP: { - TOOL: 'tool' - }, - AI: { - LLM: 'llm' - } - } - }, - getLogger: vi.fn(() => ({ - error: vi.fn() - })) -})); +vi.mock('@fastgpt/service/common/logger', () => { + const logCategories = new Proxy({}, { get: () => logCategories }); + return { + LogCategories: logCategories, + getLogger: vi.fn(() => ({ + error: vi.fn() + })) + }; +}); vi.mock('@fastgpt/service/common/middle/tracks/utils', () => ({ pushTrack: { @@ -90,7 +84,8 @@ vi.mock('@fastgpt/service/core/app/tool/systemTool/systemTool.repo', () => ({ })); vi.mock('@fastgpt/service/core/workflow/utils/context', () => ({ - getWorkflowContext: vi.fn(() => ({ mcpClientMemory: {} })) + getWorkflowContext: vi.fn(() => ({ mcpClientMemory: {} })), + getWorkflowResourceContext: vi.fn(() => undefined) })); const createRunToolProps = ( @@ -141,6 +136,13 @@ describe('dispatchRunTool runtime toolset auth', () => { modules: [] } }); + getAppVersionByIdMock.mockImplementation(({ app }: { app?: any }) => + Promise.resolve({ + nodes: app?.modules ?? [], + edges: app?.edges ?? [], + chatConfig: app?.chatConfig + }) + ); getHTTPToolListMock.mockResolvedValue([]); getMCPChildrenMock.mockResolvedValue([]); getSystemToolRuntimeMock.mockResolvedValue({ @@ -209,11 +211,10 @@ describe('dispatchRunTool runtime toolset auth', () => { ); expect(authAppByTmbIdMock).toHaveBeenCalledWith({ - tmbId: 'attacker-tmb', + tmbId: 'caller-without-toolset-permission', appId: 'victim-toolset', per: ReadPermissionVal }); - expect(getAppVersionByIdMock).not.toHaveBeenCalled(); expect(runHTTPToolMock).not.toHaveBeenCalled(); expect(result.error?.[NodeOutputKeyEnum.errorText]).toBeTruthy(); }); @@ -263,11 +264,10 @@ describe('dispatchRunTool runtime toolset auth', () => { ); expect(authAppByTmbIdMock).toHaveBeenCalledWith({ - tmbId: 'attacker-tmb', + tmbId: 'caller-without-toolset-permission', appId: 'victim-toolset', per: ReadPermissionVal }); - expect(getAppVersionByIdMock).not.toHaveBeenCalled(); expect(runHTTPToolMock).toHaveBeenCalledWith( expect.objectContaining({ baseUrl: 'https://example.com', @@ -339,7 +339,6 @@ describe('dispatchRunTool runtime toolset auth', () => { expect(rejected.error?.[NodeOutputKeyEnum.errorText]).toContain('validation failed'); } expect(runHTTPToolMock).not.toHaveBeenCalled(); - expect(getAppVersionByIdMock).not.toHaveBeenCalled(); expect(tool).toEqual(original); }); @@ -413,11 +412,10 @@ describe('dispatchRunTool runtime toolset auth', () => { ); expect(authAppByTmbIdMock).toHaveBeenCalledWith({ - tmbId: 'attacker-tmb', + tmbId: 'caller-without-toolset-permission', appId: 'victim-toolset', per: ReadPermissionVal }); - expect(getAppVersionByIdMock).not.toHaveBeenCalled(); expect(mcpToolCallMock).not.toHaveBeenCalled(); expect(result.error?.[NodeOutputKeyEnum.errorText]).toBeTruthy(); }); diff --git a/packages/service/test/core/workflow/dispatch/utils.test.ts b/packages/service/test/core/workflow/dispatch/utils.test.ts index 131d8dd7c776..daa7664001d6 100644 --- a/packages/service/test/core/workflow/dispatch/utils.test.ts +++ b/packages/service/test/core/workflow/dispatch/utils.test.ts @@ -31,6 +31,7 @@ import { } from '@fastgpt/global/core/workflow/node/constant'; import type { RuntimeEdgeItemType } from '@fastgpt/global/core/workflow/type/edge'; import type { RuntimeNodeItemType } from '@fastgpt/global/core/workflow/runtime/type'; +import { runWithContext } from '@fastgpt/service/core/workflow/utils/context'; const mockGetSystemToolRunTimeNodeFromSystemToolset = vi.fn(); vi.mock('@fastgpt/service/core/workflow/utils', () => ({ diff --git a/packages/service/test/core/workflow/utils/resource.test.ts b/packages/service/test/core/workflow/utils/resource.test.ts new file mode 100644 index 000000000000..b5ea0c145663 --- /dev/null +++ b/packages/service/test/core/workflow/utils/resource.test.ts @@ -0,0 +1,54 @@ +import { beforeEach, describe, expect, it, vi } from 'vitest'; + +const mocks = vi.hoisted(() => ({ + mongoDatasetFind: vi.fn() +})); + +vi.mock('@fastgpt/service/core/dataset/schema', async (importOriginal) => { + const actual = await importOriginal(); + return { + ...actual, + MongoDataset: { + ...actual.MongoDataset, + find: mocks.mongoDatasetFind + } + }; +}); + +import { runWithContext } from '@fastgpt/service/core/workflow/utils/context'; +import { + createWorkflowChildResourceContext, + loadWorkflowResourceContext +} from '@fastgpt/service/core/workflow/utils/resource'; + +const createFindResult = (documents: unknown[] = []) => ({ + lean: vi.fn().mockResolvedValue(documents) +}); + +describe('workflow resource context', () => { + beforeEach(() => { + vi.clearAllMocks(); + mocks.mongoDatasetFind.mockReturnValue( + createFindResult([{ _id: 'dataset-1' }, { _id: 'dataset-2' }]) + ); + }); + + it('inherits root cross-team permission when creating a child context', async () => { + const rootContext = await loadWorkflowResourceContext({ + resources: [{ type: 'dataset', id: 'dataset-1' }], + teamId: 'root-team', + isRoot: true + }); + + const childContext = await runWithContext( + { mcpClientMemory: {}, resourceContext: rootContext }, + () => createWorkflowChildResourceContext([{ type: 'dataset', id: 'dataset-2' }], 'child-team') + ); + + expect(childContext.isRoot).toBe(true); + expect(mocks.mongoDatasetFind).toHaveBeenNthCalledWith(2, { + _id: { $in: ['dataset-2'] }, + deleteTime: null + }); + }); +}); diff --git a/packages/service/test/env.test.ts b/packages/service/test/env.test.ts index 22887d46bfbb..a2af6533a7ae 100644 --- a/packages/service/test/env.test.ts +++ b/packages/service/test/env.test.ts @@ -500,8 +500,7 @@ describe('serviceEnv', () => { vi.stubEnv('AGENT_SANDBOX_PROVIDER', 'opensandbox'); vi.stubEnv('AGENT_SANDBOX_OPENSANDBOX_BASEURL', 'http://mock-opensandbox.local'); vi.stubEnv('AGENT_SANDBOX_OPENSANDBOX_API_KEY', 'mock-opensandbox-api-key'); - vi.stubEnv('AGENT_SANDBOX_OPENSANDBOX_IMAGE_REPO', 'legacy/runtime-image'); - vi.stubEnv('AGENT_SANDBOX_OPENSANDBOX_IMAGE_TAG', 'legacy-stable'); + vi.stubEnv('AGENT_SANDBOX_OPENSANDBOX_IMAGE', undefined); vi.stubEnv('AGENT_SANDBOX_OPENSANDBOX_VOLUME_MANAGER_URL', 'http://mock-volume-manager.local'); vi.stubEnv('AGENT_SANDBOX_OPENSANDBOX_VOLUME_MANAGER_TOKEN', 'mock-volume-manager-token'); diff --git a/packages/service/test/support/outLink/runtime/service.test.ts b/packages/service/test/support/outLink/runtime/service.test.ts index b306883404bc..d010c8b10ee2 100644 --- a/packages/service/test/support/outLink/runtime/service.test.ts +++ b/packages/service/test/support/outLink/runtime/service.test.ts @@ -29,6 +29,7 @@ import { mongoSessionRun } from '@fastgpt/service/common/mongo/sessionRun'; import { assertCancellation } from '@fastgpt/service/support/user/account/cancellation/guard'; vi.mock('@fastgpt/service/core/app/schema', () => ({ + AppCollectionName: 'apps', MongoApp: { findById: vi.fn() } })); vi.mock('@fastgpt/service/core/app/version/controller', () => ({ @@ -150,7 +151,8 @@ describe('runOutlinkRuntime', () => { vi.mocked(getAppLatestVersion).mockResolvedValue({ nodes: [{ nodeId: 'start', inputs: [], outputs: [] }], edges: [], - chatConfig: { variables: [], fileSelectConfig: { maxFiles: 2 } } + chatConfig: { variables: [], fileSelectConfig: { maxFiles: 2 } }, + resources: [] } as any); vi.mocked(getChatItems).mockResolvedValue({ histories: [] } as any); vi.mocked(authOutLinkLimit).mockResolvedValue({ uid: message.chatUserId }); diff --git a/packages/service/test/support/permission/controller.test.ts b/packages/service/test/support/permission/controller.test.ts index ae98f86fe3d2..2cacd6b28759 100644 --- a/packages/service/test/support/permission/controller.test.ts +++ b/packages/service/test/support/permission/controller.test.ts @@ -41,10 +41,10 @@ describe('test getClbsWithInfo', () => { const users = await getFakeUsers(3); const orgs = await getFakeOrgs(); const groups = await getFakeGroups(3); - const app = await Call(createAppAPI, { + const app = await Call, string>(createAppAPI, { auth: users.owner, body: { - modules: [], + nodes: [], name: 'test', type: AppTypeEnum.simple } diff --git a/packages/service/test/support/permission/skill/auth.test.ts b/packages/service/test/support/permission/skill/auth.test.ts index 94203d940e67..73f702728a18 100644 --- a/packages/service/test/support/permission/skill/auth.test.ts +++ b/packages/service/test/support/permission/skill/auth.test.ts @@ -1,11 +1,12 @@ -import { AgentSkillSourceEnum } from '@fastgpt/global/core/ai/skill/constants'; +import { beforeEach, describe, expect, it, vi } from 'vitest'; +import { AgentSkillSourceEnum, AgentSkillTypeEnum } from '@fastgpt/global/core/ai/skill/constants'; import { SkillErrEnum } from '@fastgpt/global/common/error/code/skill'; import { ManagePermissionVal, + OwnerPermissionVal, ReadPermissionVal, WritePermissionVal } from '@fastgpt/global/support/permission/constant'; -import { describe, expect, it, vi } from 'vitest'; const { mockFindOne, mockGetTmbInfoByTmbId, mockGetTmbPermission } = vi.hoisted(() => ({ mockFindOne: vi.fn(), @@ -29,12 +30,22 @@ import { authSkillByTmbId } from '@fastgpt/service/support/permission/skill/auth const skillId = '507f1f77bcf86cd799439011'; +const systemSkill = { + _id: 'system-skill', + source: AgentSkillSourceEnum.system, + type: AgentSkillTypeEnum.skill, + name: 'System skill', + description: '', + teamId: null, + tmbId: null, + inheritPermission: true +}; + const mockSkillQuery = (skill: Record | null) => { mockFindOne.mockReturnValue({ lean: vi.fn().mockResolvedValue(skill) }); }; const setup = () => { - vi.clearAllMocks(); mockGetTmbInfoByTmbId.mockResolvedValue({ teamId: 'team-a', permission: { isOwner: false } @@ -50,9 +61,12 @@ const setup = () => { }; describe('authSkillByTmbId', () => { - it('uses resource ACL roles for personal skills', async () => { + beforeEach(() => { + vi.clearAllMocks(); setup(); + }); + it('uses resource ACL roles for personal skills', async () => { await expect( authSkillByTmbId({ tmbId: 'member-tmb', skillId, per: ReadPermissionVal }) ).resolves.toMatchObject({ skill: { permission: { hasReadPer: true, isOwner: false } } }); @@ -62,7 +76,6 @@ describe('authSkillByTmbId', () => { }); it('falls back to the parent ACL when inheritance is missing', async () => { - setup(); mockSkillQuery({ _id: skillId, teamId: 'team-a', @@ -78,27 +91,29 @@ describe('authSkillByTmbId', () => { ).resolves.toMatchObject({ skill: { permission: { hasReadPer: true } } }); }); - it('allows system skills to be read and rejects write access', async () => { - setup(); - mockSkillQuery({ - _id: skillId, - teamId: 'team-a', - tmbId: 'system-tmb', - source: AgentSkillSourceEnum.system, - deleteTime: null - }); + it('allows team members to read global system skills but rejects writes', async () => { + mockSkillQuery(systemSkill); await expect( - authSkillByTmbId({ tmbId: 'member-tmb', skillId, per: ReadPermissionVal }) - ).resolves.toMatchObject({ skill: { permission: { hasReadPer: true } } }); + authSkillByTmbId({ + tmbId: 'member-tmb', + skillId: String(systemSkill._id), + per: ReadPermissionVal + }) + ).resolves.toMatchObject({ + skill: { _id: systemSkill._id, permission: { hasReadPer: true, isOwner: false } } + }); await expect( - authSkillByTmbId({ tmbId: 'member-tmb', skillId, per: WritePermissionVal }) + authSkillByTmbId({ + tmbId: 'member-tmb', + skillId: String(systemSkill._id), + per: OwnerPermissionVal + }) ).rejects.toBe(SkillErrEnum.unAuthSkill); expect(mockGetTmbPermission).not.toHaveBeenCalled(); }); it('allows root access and rejects cross-team access', async () => { - setup(); mockSkillQuery({ _id: skillId, teamId: 'team-b', @@ -113,10 +128,10 @@ describe('authSkillByTmbId', () => { await expect( authSkillByTmbId({ tmbId: 'member-tmb', skillId, per: ReadPermissionVal }) ).rejects.toBe(SkillErrEnum.unAuthSkill); + expect(mockGetTmbPermission).not.toHaveBeenCalled(); }); it('treats deleted skills as nonexistent', async () => { - setup(); mockSkillQuery(null); await expect( diff --git a/packages/web/i18n/en/common.json b/packages/web/i18n/en/common.json index 81bff3450a62..16382680000a 100644 --- a/packages/web/i18n/en/common.json +++ b/packages/web/i18n/en/common.json @@ -669,6 +669,8 @@ "core.workflow.check.tool_missing": "This tool does not exist, please delete it", "core.workflow.check.tool_no_permission": "Current account has no permission to access this resource", "core.workflow.check.model_unavailable": "Model has been disabled", + "core.workflow.check.resource_missing": "Referenced knowledge base or skill has been deleted or is unavailable, please remove it", + "core.workflow.check.resource_no_permission": "No permission to access this resource, please check permissions", "core.workflow.check.user_select_empty": "Configure at least one option", "core.workflow.check.user_select_value_empty": "Option cannot be empty", "core.workflow.debug.Done": "Debugging Completed", diff --git a/packages/web/i18n/ko-KR/common.json b/packages/web/i18n/ko-KR/common.json index ccc79fa24cec..8511484685d9 100644 --- a/packages/web/i18n/ko-KR/common.json +++ b/packages/web/i18n/ko-KR/common.json @@ -667,6 +667,8 @@ "core.workflow.check.tool_missing": "이 도구는 존재하지 않습니다. 삭제하세요", "core.workflow.check.tool_no_permission": "현재 계정에 이 리소스에 접근할 권한이 없습니다", "core.workflow.check.model_unavailable": "모델이 비활성화되었습니다", + "core.workflow.check.resource_missing": "참조한 지식 베이스 또는 스킬이 삭제되었거나 사용할 수 없습니다. 삭제해 주세요", + "core.workflow.check.resource_no_permission": "이 리소스에 접근할 권한이 없습니다. 권한을 확인해 주세요", "core.workflow.check.user_select_empty": "옵션을 하나 이상 설정하세요", "core.workflow.check.user_select_value_empty": "옵션 값은 비워 둘 수 없습니다", "core.workflow.debug.Done": "디버깅 완료", diff --git a/packages/web/i18n/zh-CN/common.json b/packages/web/i18n/zh-CN/common.json index a86c5212ec74..bb364b220893 100644 --- a/packages/web/i18n/zh-CN/common.json +++ b/packages/web/i18n/zh-CN/common.json @@ -669,6 +669,8 @@ "core.workflow.check.tool_missing": "该工具不存在,请删除", "core.workflow.check.tool_no_permission": "当前账号无权限访问该资源", "core.workflow.check.model_unavailable": "模型已停用", + "core.workflow.check.resource_missing": "引用的知识库或技能已删除或不可用,请删除", + "core.workflow.check.resource_no_permission": "无权限访问该资源,请检查权限", "core.workflow.check.user_select_empty": "需配置至少一个选项", "core.workflow.check.user_select_value_empty": "选项不可为空", "core.workflow.debug.Done": "完成调试", diff --git a/packages/web/i18n/zh-Hant/common.json b/packages/web/i18n/zh-Hant/common.json index a76cb6dacaf3..ac3c06cafeac 100644 --- a/packages/web/i18n/zh-Hant/common.json +++ b/packages/web/i18n/zh-Hant/common.json @@ -669,6 +669,8 @@ "core.workflow.check.tool_missing": "該工具不存在,請刪除", "core.workflow.check.tool_no_permission": "當前帳號無權限存取該資源", "core.workflow.check.model_unavailable": "模型已停用", + "core.workflow.check.resource_missing": "引用的知識庫或技能已刪除或不可用,請刪除", + "core.workflow.check.resource_no_permission": "無權限存取該資源,請檢查權限", "core.workflow.check.user_select_empty": "需配置至少一個選項", "core.workflow.check.user_select_value_empty": "選項不可為空", "core.workflow.debug.Done": "除錯完成", diff --git a/projects/app/src/components/core/app/DatasetCard.tsx b/projects/app/src/components/core/app/DatasetCard.tsx index d2e5821aa53c..4364337dc3ad 100644 --- a/projects/app/src/components/core/app/DatasetCard.tsx +++ b/projects/app/src/components/core/app/DatasetCard.tsx @@ -4,6 +4,8 @@ import { useRouter } from 'next/router'; import { useTranslation } from 'next-i18next'; import Avatar from '@fastgpt/web/components/common/Avatar'; import MyIconButton, { MyDeleteIconButton } from '@fastgpt/web/components/common/Icon/button'; +import MyIcon from '@fastgpt/web/components/common/Icon'; +import MyTag from '@fastgpt/web/components/common/Tag/index'; import type { SelectedDatasetType } from '@fastgpt/global/core/workflow/type/io'; type DatasetCardProps = { @@ -26,7 +28,7 @@ const cardProps: FlexProps = { }; /** - * 单个已选知识库卡片,仅消费后端补齐的 isDeleted 状态来展示正常态或删除态。 + * 单个已选知识库卡片,展示后端补齐的删除态和当前操作者无权限态。 */ const DatasetCard = React.memo(function DatasetCard({ dataset, @@ -36,7 +38,9 @@ const DatasetCard = React.memo(function DatasetCard({ const { t } = useTranslation(); const router = useRouter(); const isDeleted = !!dataset.isDeleted; - const hasPreviewButton = !isDeleted; + const permissionDenied = !!dataset.permissionDenied; + const isUnavailable = isDeleted || permissionDenied; + const hasPreviewButton = !isUnavailable; const hasDeleteButton = !!onDelete; const hasController = hasPreviewButton || hasDeleteButton; @@ -48,10 +52,10 @@ const DatasetCard = React.memo(function DatasetCard({ {...cardProps} {...flexProps} border={flexProps?.border || cardProps.border} - borderColor={isDeleted ? 'red.600' : flexProps?.borderColor} + borderColor={isUnavailable ? 'red.600' : flexProps?.borderColor} _hover={{ ...flexProps?._hover, - borderColor: isDeleted ? 'red.600' : 'primary.300', + borderColor: isUnavailable ? 'red.600' : 'primary.300', '& .dataset-card-controller': { opacity: 1, pointerEvents: 'auto' @@ -66,11 +70,20 @@ const DatasetCard = React.memo(function DatasetCard({ minW={0} className={'textEllipsis'} fontSize={'sm'} - color={isDeleted ? 'red.600' : 'myGray.900'} + color={isUnavailable ? 'red.600' : 'myGray.900'} > {isDeleted ? t('common:dataset_deleted') : dataset.name} + {permissionDenied && ( + + + + {t('common:core.workflow.check.resource_no_permission')} + + + )} + {hasController && ( v.setCiteModalData); const [workflowData, setWorkflowData] = useSafeState({ - nodes: appDetail.modules || [], + nodes: appDetail.nodes || [], edges: appDetail.edges || [] }); diff --git a/projects/app/src/pageComponents/app/detail/Edit/ChatAgent/EditForm.tsx b/projects/app/src/pageComponents/app/detail/Edit/ChatAgent/EditForm.tsx index 72ecbac44eaa..a4ed11e06dbb 100644 --- a/projects/app/src/pageComponents/app/detail/Edit/ChatAgent/EditForm.tsx +++ b/projects/app/src/pageComponents/app/detail/Edit/ChatAgent/EditForm.tsx @@ -428,11 +428,19 @@ const EditForm = ({ > {selectedAgentSkills.map((item) => { const isDeleted = !!item.isDeleted; + const permissionDenied = !!item.permissionDenied; + const isUnavailable = isDeleted || permissionDenied; return ( {item.name} - {isDeleted && ( + {permissionDenied && ( + + + + {t('common:core.workflow.check.resource_no_permission')} + + + )} + {!permissionDenied && isDeleted && ( @@ -757,7 +773,8 @@ const EditForm = ({ name: item.name, avatar: item.avatar, vectorModel: item.vectorModel, - isDeleted: item.isDeleted + isDeleted: item.isDeleted, + permissionDenied: item.permissionDenied }))} onClose={onCloseKbSelect} onChange={(e) => { diff --git a/projects/app/src/pageComponents/app/detail/Edit/ChatAgent/hooks/useSkillManager.tsx b/projects/app/src/pageComponents/app/detail/Edit/ChatAgent/hooks/useSkillManager.tsx index 9aaae0d17e5c..32cbdeb6dba2 100644 --- a/projects/app/src/pageComponents/app/detail/Edit/ChatAgent/hooks/useSkillManager.tsx +++ b/projects/app/src/pageComponents/app/detail/Edit/ChatAgent/hooks/useSkillManager.tsx @@ -64,7 +64,7 @@ const toSkillLabelItem = ( ...tool, id: getSkillId(tool.pluginId, tool.source), name: tool.name, - configStatus + configStatus: tool.pluginData?.permissionDenied ? 'invalid' : configStatus }); const toAgentSkillItem = (item: AgentSkillListItemType): SkillItemType => { @@ -86,7 +86,7 @@ const toAgentSkillLabelItem = (skill: SelectedAgentSkillItemType): SkillLabelIte avatar: skill.avatar || 'core/skill/default', intro: skill.description, flowNodeType: FlowNodeTypeEnum.tool, - configStatus: skill.isDeleted ? 'invalid' : 'noConfig' + configStatus: skill.isDeleted || skill.permissionDenied ? 'invalid' : 'noConfig' }); export const useSkillManager = ({ @@ -307,7 +307,8 @@ export const useSkillManager = ({ name: targetSkill.name, description: targetSkill.description, avatar: targetSkill.avatar, - isDeleted: false + isDeleted: false, + permissionDenied: false }; if (!onAddAgentSkill?.(selectedSkill)) return; const skill = toAgentSkillLabelItem(selectedSkill); @@ -513,7 +514,7 @@ export const useSkillManager = ({ const selectedSkills = useMemoEnhance(() => { const tools = selectedTools.map((tool) => { const configStatus: SkillLabelItemType['configStatus'] = (() => { - if (tool.pluginData?.error) { + if (tool.pluginData?.error || tool.pluginData?.permissionDenied) { return 'invalid'; } if (tool.pluginId === SubAppIds.datasetSearch) { diff --git a/projects/app/src/pageComponents/app/detail/Edit/ChatAgent/index.tsx b/projects/app/src/pageComponents/app/detail/Edit/ChatAgent/index.tsx index 2e56834d23d0..36c39da2311e 100644 --- a/projects/app/src/pageComponents/app/detail/Edit/ChatAgent/index.tsx +++ b/projects/app/src/pageComponents/app/detail/Edit/ChatAgent/index.tsx @@ -27,7 +27,7 @@ const AgentEdit = () => { const [appForm, setAppForm] = useState(() => { if (past.length === 0) { return appWorkflow2AgentForm({ - nodes: appDetail.modules, + nodes: appDetail.nodes, chatConfig: { ...appDetail.chatConfig, fileSelectConfig: appDetail.chatConfig.fileSelectConfig || { diff --git a/projects/app/src/pageComponents/app/detail/Edit/FormComponent/ToolSelector/SkillSelectModal.tsx b/projects/app/src/pageComponents/app/detail/Edit/FormComponent/ToolSelector/SkillSelectModal.tsx index 47061891ac9d..65c8d33ad48b 100644 --- a/projects/app/src/pageComponents/app/detail/Edit/FormComponent/ToolSelector/SkillSelectModal.tsx +++ b/projects/app/src/pageComponents/app/detail/Edit/FormComponent/ToolSelector/SkillSelectModal.tsx @@ -155,7 +155,8 @@ const SkillSelectModal = ({ name: detail.name, description: detail.description, avatar: detail.avatar, - isDeleted: false + isDeleted: false, + permissionDenied: false }); }; @@ -263,6 +264,7 @@ const SkillSelectModal = ({ name: item.name, description: item.description, avatar: item.avatar, + permissionDenied: false, isDeleted: false }) } diff --git a/projects/app/src/pageComponents/app/detail/Edit/FormComponent/ToolSelector/ToolSelect.tsx b/projects/app/src/pageComponents/app/detail/Edit/FormComponent/ToolSelector/ToolSelect.tsx index 87fdacc7ecea..6bb4ebfaf760 100644 --- a/projects/app/src/pageComponents/app/detail/Edit/FormComponent/ToolSelector/ToolSelect.tsx +++ b/projects/app/src/pageComponents/app/detail/Edit/FormComponent/ToolSelector/ToolSelect.tsx @@ -95,6 +95,8 @@ const ToolSelect = ({ const toolError = formatToolError(item.pluginData?.error) || (isOffline ? 'common:error.tool_not_exist' : undefined); + const permissionDenied = !!item.pluginData?.permissionDenied; + const hasToolError = !!toolError || permissionDenied; const isUnconfigured = item.configStatus === 'waitingForConfig'; const isDebugTool = isDebugToolSource(item.source); @@ -102,7 +104,11 @@ const ToolSelect = ({ return ( )} + {permissionDenied && ( + + + + {t('common:core.workflow.check.resource_no_permission')} + + + )} {toolError && ( @@ -163,7 +177,7 @@ const ToolSelect = ({ )} {isDebugTool && } - {!toolError && ( + {!hasToolError && ( { const { isPc } = useSystem(); const appDetail = useContextSelector(AppContext, (v) => v.appDetail); const toolSetData = useMemo(() => { - const toolSetNode = appDetail.modules.find( + const toolSetNode = appDetail.nodes.find( (item) => item.flowNodeType === FlowNodeTypeEnum.toolSet ); - const toolSet = toolSetNode?.toolConfig?.httpToolSet; - return toolSet && !('toolId' in toolSet) ? toolSet : undefined; - }, [appDetail.modules]); + return toolSetNode?.toolConfig?.httpToolSet; + }, [appDetail.nodes]); const [currentTool, setCurrentTool] = useState( toolSetData?.toolList?.[0] @@ -41,6 +40,8 @@ const Edit = () => { useEffect(() => { if (!currentTool || toolList.length === 0) { + // 组件加载或工具集更新后,将当前选择同步到可用工具列表。 + // eslint-disable-next-line react-hooks/set-state-in-effect setCurrentTool(toolList[0]); return; } diff --git a/projects/app/src/pageComponents/app/detail/Edit/HTTPTools/ManualToolModal.tsx b/projects/app/src/pageComponents/app/detail/Edit/HTTPTools/ManualToolModal.tsx index e6622fa830ac..7db9f9d00967 100644 --- a/projects/app/src/pageComponents/app/detail/Edit/HTTPTools/ManualToolModal.tsx +++ b/projects/app/src/pageComponents/app/detail/Edit/HTTPTools/ManualToolModal.tsx @@ -194,7 +194,7 @@ const ManualToolModal = ({ headerSecret: data.headerSecret }; - const toolSetNode = appDetail.modules.find( + const toolSetNode = appDetail.nodes.find( (item) => item.flowNodeType === FlowNodeTypeEnum.toolSet ); const toolSet = toolSetNode?.toolConfig?.httpToolSet; diff --git a/projects/app/src/pageComponents/app/detail/Edit/HTTPTools/SchemaConfigModal.tsx b/projects/app/src/pageComponents/app/detail/Edit/HTTPTools/SchemaConfigModal.tsx index 79c49e718094..e3649bad5469 100644 --- a/projects/app/src/pageComponents/app/detail/Edit/HTTPTools/SchemaConfigModal.tsx +++ b/projects/app/src/pageComponents/app/detail/Edit/HTTPTools/SchemaConfigModal.tsx @@ -57,12 +57,11 @@ const SchemaConfigModal = ({ onClose }: { onClose: () => void }) => { const reloadApp = useContextSelector(AppContext, (v) => v.reloadApp); const toolSetData = useMemo(() => { - const toolSetNode = appDetail.modules.find( + const toolSetNode = appDetail.nodes.find( (item) => item.flowNodeType === FlowNodeTypeEnum.toolSet ); - const toolSet = toolSetNode?.toolConfig?.httpToolSet; - return toolSet && !('toolId' in toolSet) ? toolSet : undefined; - }, [appDetail.modules]); + return toolSetNode?.toolConfig?.httpToolSet; + }, [appDetail.nodes]); const { register, setValue, handleSubmit, watch } = useForm({ defaultValues: { diff --git a/projects/app/src/pageComponents/app/detail/Edit/MCPTools/index.tsx b/projects/app/src/pageComponents/app/detail/Edit/MCPTools/index.tsx index 2cc4fd945e64..9bad33dcb19f 100644 --- a/projects/app/src/pageComponents/app/detail/Edit/MCPTools/index.tsx +++ b/projects/app/src/pageComponents/app/detail/Edit/MCPTools/index.tsx @@ -11,11 +11,11 @@ import { type StoreSecretValueType } from '@fastgpt/global/common/secret/type'; const MCPTools = () => { const appDetail = useContextSelector(AppContext, (v) => v.appDetail); const toolSetData = useMemo(() => { - const toolSetNode = appDetail.modules.find( + const toolSetNode = appDetail.nodes.find( (item) => item.flowNodeType === FlowNodeTypeEnum.toolSet ); return toolSetNode?.toolConfig?.mcpToolSet ?? toolSetNode?.inputs[0].value; - }, [appDetail.modules]); + }, [appDetail.nodes]); const [url, setUrl] = useState(toolSetData?.url || ''); const [toolList, setToolList] = useState(toolSetData?.toolList || []); diff --git a/projects/app/src/pageComponents/app/detail/Edit/SimpleApp/ChatTest.tsx b/projects/app/src/pageComponents/app/detail/Edit/SimpleApp/ChatTest.tsx index 95663e1f520b..9d437e592f19 100644 --- a/projects/app/src/pageComponents/app/detail/Edit/SimpleApp/ChatTest.tsx +++ b/projects/app/src/pageComponents/app/detail/Edit/SimpleApp/ChatTest.tsx @@ -34,7 +34,7 @@ const ChatTest = ({ appForm, setRenderEdit, form2WorkflowFn }: Props) => { const setCiteModalData = useContextSelector(ChatItemContext, (v) => v.setCiteModalData); const [workflowData, setWorkflowData] = useSafeState({ - nodes: appDetail.modules || [], + nodes: appDetail.nodes || [], edges: appDetail.edges || [] }); diff --git a/projects/app/src/pageComponents/app/detail/Edit/SimpleApp/EditForm.tsx b/projects/app/src/pageComponents/app/detail/Edit/SimpleApp/EditForm.tsx index a939e094d6a8..dde53801c6c1 100644 --- a/projects/app/src/pageComponents/app/detail/Edit/SimpleApp/EditForm.tsx +++ b/projects/app/src/pageComponents/app/detail/Edit/SimpleApp/EditForm.tsx @@ -614,7 +614,8 @@ const EditForm = ({ name: item.name, avatar: item.avatar, vectorModel: item.vectorModel, - isDeleted: item.isDeleted + isDeleted: item.isDeleted, + permissionDenied: item.permissionDenied }))} onClose={onCloseDatasetSelect} onChange={(e) => { diff --git a/projects/app/src/pageComponents/app/detail/Edit/SimpleApp/index.tsx b/projects/app/src/pageComponents/app/detail/Edit/SimpleApp/index.tsx index f88d4714ff43..1f47909498ea 100644 --- a/projects/app/src/pageComponents/app/detail/Edit/SimpleApp/index.tsx +++ b/projects/app/src/pageComponents/app/detail/Edit/SimpleApp/index.tsx @@ -23,7 +23,7 @@ const SimpleEdit = () => { const [appForm, setAppForm] = useState(() => { if (past.length === 0) { return appWorkflow2Form({ - nodes: appDetail.modules, + nodes: appDetail.nodes, chatConfig: { ...appDetail.chatConfig, fileSelectConfig: appDetail.chatConfig?.fileSelectConfig || { diff --git a/projects/app/src/pageComponents/app/detail/InfoModal.tsx b/projects/app/src/pageComponents/app/detail/InfoModal.tsx index 7dc440f81f6b..1343c89861c2 100644 --- a/projects/app/src/pageComponents/app/detail/InfoModal.tsx +++ b/projects/app/src/pageComponents/app/detail/InfoModal.tsx @@ -13,7 +13,7 @@ import { ModalFooter, Textarea } from '@chakra-ui/react'; -import type { AppSchemaType } from '@fastgpt/global/core/app/type'; +import type { AppDetailType } from '@fastgpt/global/core/app/type'; import { AppRoleList } from '@fastgpt/global/support/permission/app/constant'; import Avatar from '@fastgpt/web/components/common/Avatar'; import MyIcon from '@fastgpt/web/components/common/Icon'; @@ -51,7 +51,7 @@ const InfoModal = ({ onClose }: { onClose: () => void }) => { // submit config const { runAsync: saveSubmitSuccess, loading: btnLoading } = useRequest( - async (data: AppSchemaType) => { + async (data: AppDetailType) => { await updateAppDetail({ name: data.name, avatar: data.avatar, diff --git a/projects/app/src/pageComponents/app/detail/Plugin/index.tsx b/projects/app/src/pageComponents/app/detail/Plugin/index.tsx index 0a8332668bf8..2890c0804945 100644 --- a/projects/app/src/pageComponents/app/detail/Plugin/index.tsx +++ b/projects/app/src/pageComponents/app/detail/Plugin/index.tsx @@ -24,7 +24,7 @@ const WorkflowEdit = () => { useMount(() => { initData( cloneDeep({ - nodes: appDetail.modules || [], + nodes: appDetail.nodes || [], edges: appDetail.edges || [] }), true diff --git a/projects/app/src/pageComponents/app/detail/Workflow/index.tsx b/projects/app/src/pageComponents/app/detail/Workflow/index.tsx index 0e1f2f86056a..72a78ac0ca18 100644 --- a/projects/app/src/pageComponents/app/detail/Workflow/index.tsx +++ b/projects/app/src/pageComponents/app/detail/Workflow/index.tsx @@ -26,7 +26,7 @@ const WorkflowEdit = () => { useMount(() => { initData( cloneDeep({ - nodes: appDetail.modules || [], + nodes: appDetail.nodes || [], edges: appDetail.edges || [] }), true diff --git a/projects/app/src/pageComponents/app/detail/WorkflowComponents/context/workflowUtilsContext.tsx b/projects/app/src/pageComponents/app/detail/WorkflowComponents/context/workflowUtilsContext.tsx index cf277235c407..1a849322d302 100644 --- a/projects/app/src/pageComponents/app/detail/WorkflowComponents/context/workflowUtilsContext.tsx +++ b/projects/app/src/pageComponents/app/detail/WorkflowComponents/context/workflowUtilsContext.tsx @@ -2,6 +2,7 @@ import React, { type ReactNode, useCallback, useEffect, useMemo } from 'react'; import { createContext, useContextSelector } from 'use-context-selector'; import { useReactFlow } from 'reactflow'; +import type { Node, Edge } from 'reactflow'; import { useTranslation } from 'next-i18next'; import { useToast } from '@fastgpt/web/hooks/useToast'; import { storeNode2FlowNode, storeEdge2RenderEdge } from '@/web/core/workflow/utils'; @@ -332,12 +333,33 @@ export const WorkflowUtilsProvider = ({ children }: { children: ReactNode }) => ) || []; const edges = workflow.edges.map((item) => storeEdge2RenderEdge({ edge: item })); + // 进入编辑器后延迟校验并定位到第一个错误节点;ReactFlow 渲染后再 fitView 才有效。 + const scheduleEntryCheck = (checkNodes: Node[], checkEdges: Edge[]) => { + if (!isInit) return; + window.setTimeout(() => { + const { issueMap, hasError, firstErrorNodeId } = checkWorkflowBeforeRunOrPublish({ + nodes: checkNodes, + edges: checkEdges, + t + }); + onSyncWorkflowCheckIssues(issueMap); + if (hasError && firstErrorNodeId) { + const errorNode = checkNodes.find((node) => node.data.nodeId === firstErrorNodeId); + onUpdateNodeError(firstErrorNodeId, true); + if (errorNode) { + fitView({ nodes: [errorNode], padding: 0.3 }); + } + } + }, 300); + }; + // 有历史记录,直接用历史记录覆盖 if (isInit && past.length > 0) { const firstPast = past[0]; setNodes(firstPast.nodes); setEdges(firstPast.edges); setAppDetail((state) => ({ ...state, chatConfig: firstPast.chatConfig })); + scheduleEntryCheck(firstPast.nodes, firstPast.edges); return; } // 初始化一个历史记录 @@ -357,8 +379,21 @@ export const WorkflowUtilsProvider = ({ children }: { children: ReactNode }) => setNodes(nodes); setEdges(edges); setAppDetail((state) => ({ ...state, chatConfig: workflow.chatConfig })); + scheduleEntryCheck(nodes, edges); }, - [appDetail.chatConfig, llmModelList, past, setAppDetail, setEdges, setNodes, setPast, t] + [ + appDetail.chatConfig, + llmModelList, + past, + setAppDetail, + setEdges, + setNodes, + setPast, + t, + fitView, + onSyncWorkflowCheckIssues, + onUpdateNodeError + ] ); const contextValue = useMemo(() => { diff --git a/projects/app/src/pageComponents/app/detail/context.tsx b/projects/app/src/pageComponents/app/detail/context.tsx index e71adaa149e2..5f35d79e56c8 100644 --- a/projects/app/src/pageComponents/app/detail/context.tsx +++ b/projects/app/src/pageComponents/app/detail/context.tsx @@ -167,11 +167,6 @@ const AppContextProvider = ({ children }: { children: ReactNode }) => { return Promise.reject(new ToastHandledError('Debug tool cannot be published')); } await postPublishApp(appId, data); - setAppDetail((state) => ({ - ...state, - ...data, - modules: data.nodes || state.modules - })); reloadAppLatestVersion(); } catch (error: any) { if (error.statusText == AppErrEnum.unExist) { diff --git a/projects/app/src/pageComponents/dashboard/agent/JsonImportModal.tsx b/projects/app/src/pageComponents/dashboard/agent/JsonImportModal.tsx index 4b3f6d2b8ef6..fcf5207cb88b 100644 --- a/projects/app/src/pageComponents/dashboard/agent/JsonImportModal.tsx +++ b/projects/app/src/pageComponents/dashboard/agent/JsonImportModal.tsx @@ -163,7 +163,7 @@ const JsonImportModal = ({ scene, onClose }: JsonImportModalProps) => { name: (name || '').trim() || t('app:unnamed_app'), intro: (intro || '').trim(), type: appType, - modules: workflow.nodes, + nodes: workflow.nodes, edges: workflow.edges || [], chatConfig: workflow.chatConfig, utmParams: getUtmParams() diff --git a/projects/app/src/pageComponents/dashboard/agent/TemplateCreatePanel.tsx b/projects/app/src/pageComponents/dashboard/agent/TemplateCreatePanel.tsx index 55f073206fbc..f6584d0868da 100644 --- a/projects/app/src/pageComponents/dashboard/agent/TemplateCreatePanel.tsx +++ b/projects/app/src/pageComponents/dashboard/agent/TemplateCreatePanel.tsx @@ -88,7 +88,7 @@ const TemplateCreatePanel = ({ type }: { type: AppTypeEnum | 'all' }) => { avatar: templateDetail.avatar, name: templateDetail.name, type: appType, - modules: templateDetail.workflow.nodes || [], + nodes: templateDetail.workflow.nodes || [], edges: templateDetail.workflow.edges || [], chatConfig: templateDetail.workflow.chatConfig || {}, templateId: templateDetail.templateId diff --git a/projects/app/src/pages/api/admin/4163/initAppResources.ts b/projects/app/src/pages/api/admin/4163/initAppResources.ts new file mode 100644 index 000000000000..fcb224784068 --- /dev/null +++ b/projects/app/src/pages/api/admin/4163/initAppResources.ts @@ -0,0 +1,790 @@ +import { NextAPI } from '@/service/middleware/entry'; +import type { ApiRequestProps } from '@fastgpt/next/type'; +import { BoolSchema, IntSchema } from '@fastgpt/global/common/zod'; +import { AppFolderTypeList } from '@fastgpt/global/core/app/constants'; +import { AppResourcesSchema, type AppResourcesType } from '@fastgpt/global/core/app/type'; +import { migrateWorkflowToCurrent } from '@fastgpt/global/core/workflow/migration'; +import { decodeToolSetNodesFromStorage } from '@fastgpt/service/core/app/jsonSchemaStorage'; +import { resolveStoredAppResources, getLegacySkillIds } from '@fastgpt/service/core/app/resources'; +import { parseApiInput } from '@fastgpt/service/common/zod/requestParseError'; +import { MongoApp } from '@fastgpt/service/core/app/schema'; +import { MongoAppVersion } from '@fastgpt/service/core/app/version/schema'; +import { Types } from '@fastgpt/service/common/mongo'; +import { authCert } from '@fastgpt/service/support/permission/auth/common'; +import { isDeepStrictEqual } from 'node:util'; +import z from 'zod'; + +/* + * API: 初始化 App 资源快照 + * Route: POST /api/admin/4163/initAppResources + * Method: POST + * Description: 回填 Version.resources 与 App 正式版本指针,并清理历史 resourceRefs 与 App 图字段。 + * Tags: ['Admin', 'DataClean', 'App', 'Write'] + */ + +type LegacyResourceRefs = { + skillIds?: unknown; +}; + +const DEFAULT_BATCH_SIZE = 500; +const DEFAULT_WRITE_BATCH_SIZE = 50; + +type RawWorkflowRecord = { + _id?: Types.ObjectId; + nodes?: unknown; + modules?: unknown; + edges?: unknown; + chatConfig?: unknown; + resources?: unknown; + resourceRefs?: LegacyResourceRefs; + type?: unknown; + tmbId?: unknown; + name?: unknown; +}; + +type RawVersionPointerRecord = { + _id?: Types.ObjectId; + appId?: unknown; + isPublish?: unknown; +}; + +type MongoCollection = typeof MongoApp.collection; +type MongoBulkWriteOperation = Parameters[0][number]; + +const InitAppResourcesBodySchema = z.object({ + dryRun: BoolSchema.optional().default(true), + batchSize: IntSchema.min(1).max(5000).optional().default(DEFAULT_BATCH_SIZE), + writeBatchSize: IntSchema.min(1).max(1000).optional().default(DEFAULT_WRITE_BATCH_SIZE) +}); +export type InitAppResourcesBodyType = z.infer; + +const AppResourcesMigrationStatsSchema = z.object({ + appsScanned: z.number().int().nonnegative(), + versionsScanned: z.number().int().nonnegative(), + appsUpdated: z.number().int().nonnegative(), + versionsUpdated: z.number().int().nonnegative(), + appsSkipped: z.number().int().nonnegative(), + versionsSkipped: z.number().int().nonnegative(), + legacySkillRefs: z.number().int().nonnegative(), + legacySkillMismatches: z.number().int().nonnegative() +}); +type MigrationStats = z.infer; + +const InitAppResourcesResponseSchema = z.object({ + dryRun: z.boolean(), + batchSize: z.number().int().positive(), + writeBatchSize: z.number().int().positive(), + stats: AppResourcesMigrationStatsSchema +}); +export type InitAppResourcesResponseType = z.infer; + +type MigratedRecord = Pick; + +type MigrationOperation = { + record: RawWorkflowRecord; + operation: MongoBulkWriteOperation; + createdVersionId?: Types.ObjectId; +}; + +const createStats = (): MigrationStats => ({ + appsScanned: 0, + versionsScanned: 0, + appsUpdated: 0, + versionsUpdated: 0, + appsSkipped: 0, + versionsSkipped: 0, + legacySkillRefs: 0, + legacySkillMismatches: 0 +}); + +const getWorkflowSnapshot = (record: RawWorkflowRecord, isVersion: boolean) => { + const snapshot: Record = { + edges: record.edges, + chatConfig: record.chatConfig, + 'resourceRefs.skillIds': record.resourceRefs?.skillIds + }; + snapshot[isVersion ? 'nodes' : 'modules'] = isVersion ? record.nodes : record.modules; + return snapshot; +}; + +const getSnapshotQueryValue = (value: unknown) => + value === undefined ? { $exists: false } : { $exists: true, $eq: value }; + +const getMigrationUpdateFilter = ( + record: RawWorkflowRecord, + isVersion: boolean, + pointers?: { publishedVersionId?: Types.ObjectId } +) => { + const snapshot = getWorkflowSnapshot(record, isVersion); + const filter = Object.entries(snapshot).reduce>( + (nextFilter, [key, value]) => { + nextFilter[key] = getSnapshotQueryValue(value); + return nextFilter; + }, + { _id: record._id } + ); + // 只回填仍为空或仍是本次扫描结果的正式指针,避免覆盖并发发布结果。 + const pointerFilters = [ + pointers?.publishedVersionId + ? { + $or: [ + { publishedVersionId: { $exists: false } }, + { publishedVersionId: null }, + { publishedVersionId: pointers.publishedVersionId } + ] + } + : undefined + ].filter(Boolean); + if (pointerFilters.length > 0) { + filter.$and = pointerFilters; + } + return filter; +}; + +const getSnapshotProjection = (isVersion: boolean) => + Object.keys(getWorkflowSnapshot({}, isVersion)).reduce>( + (projection, key) => { + projection[key] = 1; + return projection; + }, + { _id: 1 } + ); + +const isWorkflowSnapshotUnchanged = ({ + record, + currentRecord, + isVersion +}: { + record: RawWorkflowRecord; + currentRecord?: RawWorkflowRecord; + isVersion: boolean; +}) => { + if (!currentRecord) return false; + const expectedSnapshot = getWorkflowSnapshot(record, isVersion); + const currentSnapshot = getWorkflowSnapshot(currentRecord, isVersion); + return Object.entries(expectedSnapshot).every(([key, value]) => + isDeepStrictEqual(value, currentSnapshot[key]) + ); +}; + +const getObjectIdList = (ids: Set) => Array.from(ids, (id) => new Types.ObjectId(id)); + +const countMissingLegacySkills = ({ + legacySkillIds, + resources +}: { + legacySkillIds: string[]; + resources: AppResourcesType; +}) => { + const skillIds = new Set( + resources.filter((resource) => resource.type === 'skill').map((resource) => resource.id) + ); + return legacySkillIds.filter((id) => !skillIds.has(id)).length; +}; + +const isFolderApp = (type: unknown) => + typeof type === 'string' && + AppFolderTypeList.includes(type as (typeof AppFolderTypeList)[number]); + +/** 从原始 Mongo 记录读取工作流节点,兼容旧对象 Schema 和新版字符串 Schema。 */ +const getDecodedWorkflowNodes = ({ + nodes, + modules +}: Pick) => { + const storedNodes = Array.isArray(nodes) ? nodes : Array.isArray(modules) ? modules : []; + return decodeToolSetNodesFromStorage(storedNodes); +}; + +const buildResources = ({ + nodes, + modules, + edges, + chatConfig, + resources, + resourceRefs, + stats +}: RawWorkflowRecord & { stats: MigrationStats }): AppResourcesType => { + const workflowNodes = getDecodedWorkflowNodes({ nodes, modules }); + const legacySkillIds = getLegacySkillIds(resourceRefs); + stats.legacySkillRefs += legacySkillIds.length; + + const normalizedWorkflow = migrateWorkflowToCurrent({ + nodes: workflowNodes, + edges: Array.isArray(edges) ? edges : [], + chatConfig + }); + const resolved = resolveStoredAppResources({ + resources, + nodes: normalizedWorkflow.nodes, + chatConfig: normalizedWorkflow.chatConfig, + resourceRefs + }); + stats.legacySkillMismatches += countMissingLegacySkills({ + legacySkillIds, + resources: resolved + }); + return resolved; +}; + +/** + * 按 time 倒序选出每个 App 最新正式 Version,并记录是否存在任意 Version。 + * 最新工作 Version 或最新正式 Version 被 OCC 跳过时,整 App 的指针回填推迟到下次重试。 + */ +const getLatestVersionPointers = async ({ + collection, + batchSize, + skippedRecordIds +}: { + collection: MongoCollection; + batchSize: number; + skippedRecordIds: Set; +}) => { + const appIdsWithVersions = new Set(); + const latestPublishedVersionIds = new Map(); + const skippedAppIds = new Set(); + const seenVersionAppIds = new Set(); + const seenPublishedAppIds = new Set(); + const cursor = collection + .find({}, { projection: { _id: 1, appId: 1, isPublish: 1 } }) + .sort({ time: -1, _id: -1 }) + .batchSize(batchSize); + + for await (const rawRecord of cursor) { + const record = rawRecord as RawVersionPointerRecord; + if (record.appId === undefined || record._id === undefined) continue; + const appId = String(record.appId); + const skipped = skippedRecordIds.has(String(record._id)); + appIdsWithVersions.add(appId); + if (!seenVersionAppIds.has(appId)) { + seenVersionAppIds.add(appId); + if (skipped) skippedAppIds.add(appId); + } + + if (record.isPublish === true && !seenPublishedAppIds.has(appId)) { + seenPublishedAppIds.add(appId); + if (skipped) skippedAppIds.add(appId); + else latestPublishedVersionIds.set(appId, record._id); + } + } + + return { appIdsWithVersions, latestPublishedVersionIds, skippedAppIds }; +}; + +/** + * 按固定读取和写入批次迁移单个集合,先计算完整批次资源再批量写入; + * 更新条件只包含资源计算所需的读取快照,避免并发保存覆盖用户的新版本。 + */ +const migrateCollection = async ({ + collection, + stats, + dryRun, + isVersion, + batchSize, + writeBatchSize, + buildResources, + shouldSkipRecord +}: { + collection: MongoCollection; + stats: MigrationStats; + dryRun: boolean; + isVersion: boolean; + batchSize: number; + writeBatchSize: number; + buildResources: (record: RawWorkflowRecord) => AppResourcesType; + shouldSkipRecord?: (record: RawWorkflowRecord) => boolean; +}) => { + const skippedRecordIds = new Set(); + const migratedResources = new Map(); + + const markSkipped = (record: RawWorkflowRecord) => { + if (record._id !== undefined) skippedRecordIds.add(String(record._id)); + if (isVersion) stats.versionsSkipped += 1; + else stats.appsSkipped += 1; + }; + + const flushWrites = async (operations: MigrationOperation[]) => { + for (let start = 0; start < operations.length; start += writeBatchSize) { + const batchOperations = operations.slice(start, start + writeBatchSize); + await collection.bulkWrite( + batchOperations.map(({ operation }) => operation), + { ordered: false } + ); + + // bulkWrite 只返回批次汇总结果,写入后重新读取快照才能识别具体被并发修改的记录。 + const currentRecords = await collection + .find( + { + _id: { + $in: batchOperations + .map(({ record }) => record._id) + .filter((id): id is Types.ObjectId => id !== undefined) + } + }, + { projection: getSnapshotProjection(isVersion) } + ) + .toArray(); + const currentRecordMap = new Map( + currentRecords.map((record) => [String(record._id), record as RawWorkflowRecord]) + ); + let skippedCount = 0; + + batchOperations.forEach(({ record }) => { + if ( + !isWorkflowSnapshotUnchanged({ + record, + currentRecord: currentRecordMap.get(String(record._id)), + isVersion + }) + ) { + skippedCount += 1; + markSkipped(record); + return; + } + }); + + const updatedCount = batchOperations.length - skippedCount; + if (isVersion) stats.versionsUpdated += updatedCount; + else stats.appsUpdated += updatedCount; + } + }; + + const migrateBatch = async (records: RawWorkflowRecord[]) => { + const operations: MigrationOperation[] = []; + + records.forEach((record) => { + if (isVersion) stats.versionsScanned += 1; + else stats.appsScanned += 1; + + if (!dryRun && shouldSkipRecord?.(record)) { + stats.legacySkillRefs += getLegacySkillIds(record.resourceRefs).length; + markSkipped(record); + return; + } + + const resources = buildResources(record); + if (dryRun && record._id !== undefined) { + migratedResources.set(String(record._id), resources); + } + if (dryRun) { + return; + } + if (record._id === undefined) { + markSkipped(record); + return; + } + + operations.push({ + record, + operation: { + updateOne: { + filter: getMigrationUpdateFilter(record, isVersion), + update: { + $set: { resources } + } + } + } + }); + }); + + if (!dryRun) await flushWrites(operations); + }; + + // Version 必须保持与 getAppLatestVersion 一致的时间倒序;App 使用稳定的 _id 顺序。 + const cursor = collection + .find({}) + .sort(isVersion ? { time: -1 } : { _id: 1 }) + .batchSize(batchSize); + let records: RawWorkflowRecord[] = []; + + for await (const rawRecord of cursor) { + records.push(rawRecord as RawWorkflowRecord); + if (records.length < batchSize) continue; + + await migrateBatch(records); + records = []; + } + + if (records.length > 0) await migrateBatch(records); + + return { skippedRecordIds, migratedResources }; +}; + +/** 资源迁移和全量校验完成后,按批次清理历史 resourceRefs 字段。 */ +const cleanupLegacyResourceRefs = async ({ + collection, + batchSize, + writeBatchSize, + isVersion, + excludedRecordIds +}: { + collection: MongoCollection; + batchSize: number; + writeBatchSize: number; + isVersion: boolean; + excludedRecordIds: Set; +}) => { + const flushWrites = async (operations: MongoBulkWriteOperation[]) => { + for (let start = 0; start < operations.length; start += writeBatchSize) { + const batchOperations = operations.slice(start, start + writeBatchSize); + const result = await collection.bulkWrite(batchOperations, { ordered: false }); + const matchedCount = result.matchedCount ?? 0; + if (matchedCount !== batchOperations.length) { + throw new Error( + `Legacy resourceRefs cleanup missed ${isVersion ? 'version' : 'app'} documents: ` + + `expected=${batchOperations.length}, matched=${matchedCount}` + ); + } + } + }; + + const excludedObjectIds = getObjectIdList(excludedRecordIds); + const cursor = collection + .find( + { + resourceRefs: { $exists: true }, + ...(excludedObjectIds.length > 0 ? { _id: { $nin: excludedObjectIds } } : {}) + }, + { projection: { _id: 1 } } + ) + .sort({ _id: 1 }) + .batchSize(batchSize); + let operations: MongoBulkWriteOperation[] = []; + + for await (const rawRecord of cursor) { + operations.push({ + updateOne: { + filter: { _id: rawRecord._id }, + update: { + $unset: { resourceRefs: 1 } + } + } + }); + if (operations.length < writeBatchSize) continue; + + await flushWrites(operations); + operations = []; + } + + if (operations.length > 0) await flushWrites(operations); +}; + +const verifyResources = async ({ + collection, + collectionName, + batchSize, + skippedRecordIds +}: { + collection: MongoCollection; + collectionName: string; + batchSize: number; + skippedRecordIds: Set; +}) => { + const cursor = collection.find({}, { projection: { _id: 1, resources: 1 } }).batchSize(batchSize); + for await (const rawRecord of cursor) { + const record = rawRecord as MigratedRecord; + if (record._id !== undefined && skippedRecordIds.has(String(record._id))) continue; + if ( + !Array.isArray(record.resources) || + !AppResourcesSchema.safeParse(record.resources).success + ) { + throw new Error(`Invalid resources after migration in ${collectionName} ${record._id}`); + } + } +}; + +/** + * 回填 publishedVersionId,并 $unset App 图字段。 + * 仅当该 App 一条 Version 都没有时,才用当前 App 图补建一条正式 Version。 + */ +const backfillAppVersionPointers = async ({ + appCollection, + versionCollection, + stats, + dryRun, + batchSize, + writeBatchSize, + appIdsWithVersions, + latestPublishedVersionIds, + skippedAppIdsFromVersions +}: { + appCollection: MongoCollection; + versionCollection: MongoCollection; + stats: MigrationStats; + dryRun: boolean; + batchSize: number; + writeBatchSize: number; + appIdsWithVersions: Set; + latestPublishedVersionIds: Map; + skippedAppIdsFromVersions: Set; +}) => { + const skippedRecordIds = new Set(); + + const markSkipped = (record: RawWorkflowRecord) => { + if (record._id !== undefined) skippedRecordIds.add(String(record._id)); + stats.appsSkipped += 1; + }; + + const flushWrites = async (operations: MigrationOperation[]) => { + for (let start = 0; start < operations.length; start += writeBatchSize) { + const batchOperations = operations.slice(start, start + writeBatchSize); + await appCollection.bulkWrite( + batchOperations.map(({ operation }) => operation), + { ordered: false } + ); + + const currentRecords = await appCollection + .find( + { + _id: { + $in: batchOperations + .map(({ record }) => record._id) + .filter((id): id is Types.ObjectId => id !== undefined) + } + }, + { projection: { _id: 1, resourceRefs: 1, publishedVersionId: 1 } } + ) + .toArray(); + const currentRecordMap = new Map( + currentRecords.map((record) => [ + String(record._id), + record as RawWorkflowRecord & { + publishedVersionId?: Types.ObjectId; + } + ]) + ); + let skippedCount = 0; + + for (const { record, operation, createdVersionId } of batchOperations) { + const currentRecord = currentRecordMap.get(String(record._id)); + const expectedSet = + 'updateOne' in operation && !Array.isArray(operation.updateOne.update) + ? ( + operation.updateOne.update as { + $set?: { + publishedVersionId?: Types.ObjectId; + }; + } + ).$set + : undefined; + const pointerApplied = + !!currentRecord && + (!expectedSet?.publishedVersionId || + String(currentRecord.publishedVersionId) === String(expectedSet.publishedVersionId)); + // 指针回填会 $unset 图字段,不能再用 modules 快照做 OCC。 + if ( + !pointerApplied || + !isDeepStrictEqual(record.resourceRefs?.skillIds, currentRecord?.resourceRefs?.skillIds) + ) { + skippedCount += 1; + // 只有指针没有回填成功时才能清理本次新建 Version;指针已生效时保留它,避免产生悬空指针。 + if (!pointerApplied && createdVersionId) { + const { deletedCount } = await versionCollection.deleteOne({ + _id: createdVersionId, + appId: record._id + }); + if (deletedCount === 1) stats.versionsUpdated -= 1; + } + markSkipped(record); + } + } + + stats.appsUpdated += batchOperations.length - skippedCount; + } + }; + + const createMissingPublishedVersion = async (record: RawWorkflowRecord) => { + if (record._id === undefined || record.tmbId === undefined) return; + const workflowNodes = Array.isArray(record.modules) ? record.modules : []; + const normalizedWorkflow = migrateWorkflowToCurrent({ + nodes: decodeToolSetNodesFromStorage(workflowNodes), + edges: Array.isArray(record.edges) ? record.edges : [], + chatConfig: record.chatConfig + }); + const resources = resolveStoredAppResources({ + nodes: normalizedWorkflow.nodes, + chatConfig: normalizedWorkflow.chatConfig, + resourceRefs: record.resourceRefs + }); + const result = await versionCollection.insertOne({ + tmbId: String(record.tmbId), + appId: record._id, + time: new Date(), + nodes: normalizedWorkflow.nodes, + edges: normalizedWorkflow.edges, + chatConfig: normalizedWorkflow.chatConfig, + isPublish: true, + versionName: typeof record.name === 'string' ? record.name : undefined, + resources + }); + stats.versionsUpdated += 1; + return result.insertedId; + }; + + const migrateBatch = async (records: RawWorkflowRecord[]) => { + const operations: MigrationOperation[] = []; + + for (const record of records) { + stats.appsScanned += 1; + stats.legacySkillRefs += getLegacySkillIds(record.resourceRefs).length; + const appId = record._id === undefined ? undefined : String(record._id); + + if (!dryRun && appId && skippedAppIdsFromVersions.has(appId)) { + markSkipped(record); + continue; + } + if (dryRun) continue; + if (record._id === undefined) { + markSkipped(record); + continue; + } + + const folder = isFolderApp(record.type); + let publishedVersionId = appId ? latestPublishedVersionIds.get(appId) : undefined; + let createdVersionId: Types.ObjectId | undefined; + + // 零 Version 才用 App 图补建正式版;文件夹没有工作流。 + if (!folder && appId && !appIdsWithVersions.has(appId)) { + createdVersionId = await createMissingPublishedVersion(record); + if (!createdVersionId) { + markSkipped(record); + continue; + } + publishedVersionId = createdVersionId; + } + + const $set: Record = {}; + if (publishedVersionId) $set.publishedVersionId = publishedVersionId; + + operations.push({ + record, + createdVersionId, + operation: { + updateOne: { + filter: getMigrationUpdateFilter(record, false, { + publishedVersionId + }), + update: { + ...(Object.keys($set).length > 0 ? { $set } : {}), + $unset: { modules: 1, edges: 1, chatConfig: 1 } + } + } + } + }); + } + + if (!dryRun) await flushWrites(operations); + }; + + const cursor = appCollection.find({}).sort({ _id: 1 }).batchSize(batchSize); + let records: RawWorkflowRecord[] = []; + + for await (const rawRecord of cursor) { + records.push(rawRecord as RawWorkflowRecord); + if (records.length < batchSize) continue; + await migrateBatch(records); + records = []; + } + if (records.length > 0) await migrateBatch(records); + + return { skippedRecordIds }; +}; + +/** 管理员 App 资源迁移;默认只扫描校验,dryRun=false 时才写入数据库。 */ +export async function runInitAppResourcesMigration( + params: InitAppResourcesBodyType +): Promise { + const options = InitAppResourcesBodySchema.parse(params); + const stats = createStats(); + const versionCollection = MongoAppVersion.collection; + const appCollection = MongoApp.collection; + + const { skippedRecordIds: skippedVersionIds } = await migrateCollection({ + collection: versionCollection, + stats, + dryRun: options.dryRun, + isVersion: true, + batchSize: options.batchSize, + writeBatchSize: options.writeBatchSize, + buildResources: (record) => buildResources({ ...record, stats }) + }); + const { appIdsWithVersions, latestPublishedVersionIds, skippedAppIds } = + await getLatestVersionPointers({ + collection: versionCollection, + batchSize: options.batchSize, + skippedRecordIds: skippedVersionIds + }); + + const { skippedRecordIds: skippedAppIdsFromPointer } = await backfillAppVersionPointers({ + appCollection, + versionCollection, + stats, + dryRun: options.dryRun, + batchSize: options.batchSize, + writeBatchSize: options.writeBatchSize, + appIdsWithVersions, + latestPublishedVersionIds, + skippedAppIdsFromVersions: skippedAppIds + }); + + if (!options.dryRun) { + await verifyResources({ + collection: versionCollection, + collectionName: MongoAppVersion.collection.name, + batchSize: options.batchSize, + skippedRecordIds: skippedVersionIds + }); + + await Promise.all([ + cleanupLegacyResourceRefs({ + collection: versionCollection, + batchSize: options.batchSize, + writeBatchSize: options.writeBatchSize, + isVersion: true, + excludedRecordIds: skippedVersionIds + }), + cleanupLegacyResourceRefs({ + collection: appCollection, + batchSize: options.batchSize, + writeBatchSize: options.writeBatchSize, + isVersion: false, + excludedRecordIds: skippedAppIdsFromPointer + }) + ]); + + const [remainingApps, remainingVersions] = await Promise.all([ + appCollection.countDocuments({ resourceRefs: { $exists: true } }), + versionCollection.countDocuments({ resourceRefs: { $exists: true } }) + ]); + const skippedAppCount = await appCollection.countDocuments({ + _id: { $in: getObjectIdList(skippedAppIdsFromPointer) }, + resourceRefs: { $exists: true } + }); + const skippedVersionCount = await versionCollection.countDocuments({ + _id: { $in: getObjectIdList(skippedVersionIds) }, + resourceRefs: { $exists: true } + }); + if (remainingApps > skippedAppCount || remainingVersions > skippedVersionCount) { + throw new Error( + `resourceRefs migration incomplete: apps=${remainingApps}, versions=${remainingVersions}` + ); + } + } + + return InitAppResourcesResponseSchema.parse({ + dryRun: options.dryRun, + batchSize: options.batchSize, + writeBatchSize: options.writeBatchSize, + stats + }); +} + +async function handler(req: ApiRequestProps): Promise { + await authCert({ req, authRoot: true }); + const { body } = parseApiInput({ + req, + bodySchema: InitAppResourcesBodySchema + }); + return runInitAppResourcesMigration(body); +} + +export default NextAPI(handler); diff --git a/projects/app/src/pages/api/core/ai/skill/apps.ts b/projects/app/src/pages/api/core/ai/skill/apps.ts index c96a595c739f..cc6c23829ca5 100644 --- a/projects/app/src/pages/api/core/ai/skill/apps.ts +++ b/projects/app/src/pages/api/core/ai/skill/apps.ts @@ -1,4 +1,3 @@ -import { MongoApp } from '@fastgpt/service/core/app/schema'; import { NextAPI } from '@/service/middleware/entry'; import { PerResourceTypeEnum, @@ -16,7 +15,7 @@ import type { } from '@fastgpt/global/core/ai/skill/api'; import { ListAppsBySkillIdQuerySchema } from '@fastgpt/global/core/ai/skill/api'; import { parseApiInput } from '@fastgpt/service/common/zod/requestParseError'; -import { buildAppSkillRefMongoQuery } from '@fastgpt/service/core/app/resourceRefs'; +import { findTeamAppsByPublishedResource } from '@fastgpt/service/core/app/resourceLookup'; async function handler( req: ApiRequestProps @@ -56,17 +55,13 @@ async function handler( ); })(); - // 查询最新发布版本缓存引用该 skillId 的应用。 - const apps = await MongoApp.find( - { - teamId, - deleteTime: null, - ...buildAppSkillRefMongoQuery(skillId) - }, - '_id parentId avatar type name intro tmbId updateTime inheritPermission' - ) - .sort({ updateTime: -1 }) - .lean(); + const { apps } = await findTeamAppsByPublishedResource({ + teamId, + type: 'skill', + ids: skillId, + projection: 'parentId avatar type name intro tmbId updateTime inheritPermission' + }); + apps.sort((a, b) => +new Date(b.updateTime) - +new Date(a.updateTime)); const visibleApps = apps .filter( diff --git a/projects/app/src/pages/api/core/ai/skill/detail.ts b/projects/app/src/pages/api/core/ai/skill/detail.ts index 9ba25fde0fe3..346ad97b708c 100644 --- a/projects/app/src/pages/api/core/ai/skill/detail.ts +++ b/projects/app/src/pages/api/core/ai/skill/detail.ts @@ -8,11 +8,9 @@ import type { import { GetSkillDetailQuerySchema } from '@fastgpt/global/core/ai/skill/api'; import { isValidObjectId } from 'mongoose'; import type { ApiRequestProps } from '@fastgpt/next/type'; -import { MongoApp } from '@fastgpt/service/core/app/schema'; -import { Types } from '@fastgpt/service/common/mongo'; import { SkillErrEnum } from '@fastgpt/global/common/error/code/skill'; import { parseApiInput } from '@fastgpt/service/common/zod/requestParseError'; -import { buildAppSkillRefMongoQuery } from '@fastgpt/service/core/app/resourceRefs'; +import { findTeamAppsByPublishedResource } from '@fastgpt/service/core/app/resourceLookup'; async function handler( req: ApiRequestProps, GetSkillDetailQuery> @@ -31,11 +29,12 @@ async function handler( per: ReadPermissionVal }); - const appCount = await MongoApp.countDocuments({ - teamId: new Types.ObjectId(String(teamId)), - deleteTime: null, - ...buildAppSkillRefMongoQuery(skill._id.toString()) + const { counts } = await findTeamAppsByPublishedResource({ + teamId, + type: 'skill', + ids: skill._id.toString() }); + const appCount = counts.get(skill._id.toString()) ?? 0; return { _id: skill._id, diff --git a/projects/app/src/pages/api/core/app/copy.ts b/projects/app/src/pages/api/core/app/copy.ts index c26fdc6430ee..8a11813d7769 100644 --- a/projects/app/src/pages/api/core/app/copy.ts +++ b/projects/app/src/pages/api/core/app/copy.ts @@ -21,9 +21,9 @@ import { import { AppTypeEnum } from '@fastgpt/global/core/app/constants'; import { encodeHttpToolSetNodesForStorage, - encodeMcpToolSetNodesForStorage, - decodeToolSetNodesFromStorage + encodeMcpToolSetNodesForStorage } from '@fastgpt/service/core/app/jsonSchemaStorage'; +import { getAppDraftWorkflow } from '@fastgpt/service/core/app/version/controller'; async function handler(req: ApiRequestProps): Promise { const { appId: sourceAppId } = parseApiInput({ @@ -51,12 +51,13 @@ async function handler(req: ApiRequestProps): Promise { + const draftWorkflow = await getAppDraftWorkflow(app._id); + const storageNodes = (() => { if (app.type === AppTypeEnum.mcpToolSet) { - return encodeMcpToolSetNodesForStorage(app.modules); + return encodeMcpToolSetNodesForStorage(draftWorkflow.nodes); } if (app.type === AppTypeEnum.httpToolSet) { - return encodeHttpToolSetNodesForStorage(app.modules); + return encodeHttpToolSetNodesForStorage(draftWorkflow.nodes); } // 普通应用必须写入 onCreateApp 清洗后的 workflow,不能用原始存储数据绕过模型校验。 return undefined; @@ -67,10 +68,10 @@ async function handler(req: ApiRequestProps): Promise) { req, bodySchema: CreateAppRequestBodySchema }); - const { parentId, name, avatar, intro, type, modules, edges, chatConfig, templateId, utmParams } = + const { parentId, name, avatar, intro, type, nodes, edges, chatConfig, templateId, utmParams } = body; // 凭证校验 @@ -88,7 +90,7 @@ async function handler(req: ApiRequestProps) { avatar: avatar ?? undefined, intro: intro ?? undefined, type, - modules, + nodes, edges, chatConfig, teamId, @@ -126,8 +128,8 @@ export const onCreateApp = async ({ intro, avatar, type, - modules, - storageModules, + nodes, + storageNodes, edges, chatConfig, teamId, @@ -143,10 +145,10 @@ export const onCreateApp = async ({ name?: string; avatar?: string; type: AppTypeEnum; - modules?: unknown[]; - storageModules?: AppSchemaType['modules']; - edges?: AppSchemaType['edges']; - chatConfig?: AppSchemaType['chatConfig']; + nodes?: unknown[]; + storageNodes?: AppVersionSchemaType['nodes']; + edges?: AppVersionSchemaType['edges']; + chatConfig?: AppVersionSchemaType['chatConfig']; intro?: string; teamId: string; tmbId: string; @@ -170,7 +172,7 @@ export const onCreateApp = async ({ // Copy 和 Transition 会传入历史数据库记录;写入前统一转换为 canonical 并格式化敏感字段。 const normalizedWorkflow = migrateWorkflowToCurrent({ - nodes: modules ?? [], + nodes: nodes ?? [], edges: edges ?? [], chatConfig }); @@ -182,16 +184,19 @@ export const onCreateApp = async ({ modelReferencePolicy: 'fallback' }); await beforeUpdateAppFormat({ nodes: normalizedWorkflow.nodes, teamId }); + const resources = extractAppResources({ + nodes: normalizedWorkflow.nodes, + chatConfig: normalizedWorkflow.chatConfig + }); if (!AppFolderTypeList.includes(type!)) { - await validatePublishAppAgentSkillReadPermissions({ - nodes: normalizedWorkflow.nodes, + await checkAppResourceReadPermissions({ + resources, tmbId, isRoot }); } const create = async (session: ClientSession) => { - const resourceRefs = extractAppResourceRefsFromNodes(normalizedWorkflow.nodes); const _avatar = await (async () => { if (!templateId || isPluginSystemTemplate(templateId)) return avatar; @@ -220,14 +225,10 @@ export const onCreateApp = async ({ intro, teamId, tmbId, - modules: storageModules ?? normalizedWorkflow.nodes, - edges: normalizedWorkflow.edges, - chatConfig: normalizedWorkflow.chatConfig, type, version: 'v2', pluginData, - templateId, - ...(!AppFolderTypeList.includes(type!) && { resourceRefs }) + templateId } ], { session, ordered: true } @@ -236,23 +237,33 @@ export const onCreateApp = async ({ const appId = String(app._id); if (!AppFolderTypeList.includes(type!)) { - await MongoAppVersion.create( + const [version] = await MongoAppVersion.create( [ { tmbId, appId, - nodes: storageModules ?? normalizedWorkflow.nodes, + nodes: storageNodes ?? normalizedWorkflow.nodes, edges: normalizedWorkflow.edges, chatConfig: normalizedWorkflow.chatConfig, versionName: name, username, avatar: userAvatar, isPublish: true, - resourceRefs + resources } ], { session, ordered: true } ); + await MongoApp.updateOne( + { _id: appId }, + { + $set: { + publishedVersionId: version._id, + 'pluginData.nodeVersion': version._id + } + }, + { session } + ); } await createResourceDefaultCollaborators({ @@ -294,25 +305,30 @@ export const onCreateApp = async ({ * 将已有应用转换为 workflow 时写入其 workflow 数据。 * * 该入口只服务 Transition 的 createNew=false 分支:源 workflow 可能是历史数据,写入前统一 - * 产出 canonical 数据并格式化敏感字段。调用方必须传入同一事务的 session,普通更新接口不复用。 + * 产出 canonical 数据并格式化敏感字段。resources 直接复用当前最新 Version 的权限快照,不按 + * 转化操作者重新校验或过滤。调用方必须传入同一事务的 session,普通更新接口不复用。 */ export const onUpdateAppWorkflow = async ({ appId, - modules, + nodes, edges, chatConfig, + resources, teamId, + tmbId, session }: { appId: string; - modules?: AppSchemaType['modules']; - edges?: AppSchemaType['edges']; - chatConfig?: AppSchemaType['chatConfig']; + nodes?: AppVersionSchemaType['nodes']; + edges?: AppVersionSchemaType['edges']; + chatConfig?: AppVersionSchemaType['chatConfig']; + resources: AppResourcesType; teamId: string; + tmbId: string; session?: ClientSession; }) => { const workflow = migrateWorkflowToCurrent({ - nodes: modules ?? [], + nodes: nodes ?? [], edges: edges ?? [], chatConfig }); @@ -325,13 +341,25 @@ export const onUpdateAppWorkflow = async ({ }); await beforeUpdateAppFormat({ nodes: workflow.nodes, teamId }); - return await MongoApp.findByIdAndUpdate( - appId, + await MongoAppVersion.findOneAndUpdate( + { appId, isAutoSave: true }, { - type: AppTypeEnum.workflow, - modules: workflow.nodes, + tmbId, + appId, + isAutoSave: true, + nodes: workflow.nodes, edges: workflow.edges, chatConfig: workflow.chatConfig, + time: new Date(), + resources + }, + { session, upsert: true, new: true } + ); + + return MongoApp.findByIdAndUpdate( + appId, + { + type: AppTypeEnum.workflow, updateTime: new Date() }, { session } diff --git a/projects/app/src/pages/api/core/app/detail.ts b/projects/app/src/pages/api/core/app/detail.ts index 820e2c588347..bf67b267d72c 100644 --- a/projects/app/src/pages/api/core/app/detail.ts +++ b/projects/app/src/pages/api/core/app/detail.ts @@ -4,7 +4,6 @@ import { NextAPI } from '@/service/middleware/entry'; import { ReadPermissionVal } from '@fastgpt/global/support/permission/constant'; import { CommonErrEnum } from '@fastgpt/global/common/error/code/common'; import { rewriteAppWorkflowToDetail } from '@fastgpt/service/core/app/utils'; -import { migrateWorkflowToCurrent } from '@fastgpt/global/core/workflow/migration'; import { getLocale } from '@fastgpt/service/common/middle/i18n'; import { parseApiInput } from '@fastgpt/service/common/zod/requestParseError'; import { @@ -12,7 +11,7 @@ import { GetAppDetailResponseSchema, type GetAppDetailResponseType } from '@fastgpt/global/openapi/core/app/common/api'; -import { decodeToolSetNodesFromStorage } from '@fastgpt/service/core/app/jsonSchemaStorage'; +import { getAppDraftWorkflow } from '@fastgpt/service/core/app/version/controller'; /* 获取应用详情 */ async function handler(req: NextApiRequest): Promise { @@ -25,24 +24,22 @@ async function handler(req: NextApiRequest): Promise { Promise.reject(CommonErrEnum.missingParams); } // 凭证校验 - const { app, teamId, isRoot } = await authApp({ + const { app, teamId, tmbId, isRoot } = await authApp({ req, authToken: true, appId, per: ReadPermissionVal }); - const workflow = migrateWorkflowToCurrent({ - nodes: decodeToolSetNodesFromStorage(app.modules), - edges: app.edges, - chatConfig: app.chatConfig - }); + const workflow = await getAppDraftWorkflow(app._id); await rewriteAppWorkflowToDetail({ nodes: workflow.nodes, teamId, + viewerTmbId: tmbId, ownerTmbId: app.tmbId, isRoot, - lang: getLocale(req) + lang: getLocale(req), + resources: workflow.resources }); if (!app.permission.hasWritePer) { @@ -50,8 +47,9 @@ async function handler(req: NextApiRequest): Promise { ...app, avatar: app.avatar ?? '', intro: app.intro ?? '', - modules: [], - edges: [] + nodes: [], + edges: [], + chatConfig: workflow.chatConfig }); } @@ -59,7 +57,7 @@ async function handler(req: NextApiRequest): Promise { ...app, avatar: app.avatar ?? '', intro: app.intro ?? '', - modules: workflow.nodes, + nodes: workflow.nodes, edges: workflow.edges, chatConfig: workflow.chatConfig }); diff --git a/projects/app/src/pages/api/core/app/httpTools/create.ts b/projects/app/src/pages/api/core/app/httpTools/create.ts index 2d406cd43c4f..8fd150076fd6 100644 --- a/projects/app/src/pages/api/core/app/httpTools/create.ts +++ b/projects/app/src/pages/api/core/app/httpTools/create.ts @@ -58,8 +58,8 @@ async function handler( teamId, tmbId, type: AppTypeEnum.httpToolSet, - modules: [toolSetRuntimeNode], - storageModules: encodeHttpToolSetNodesForStorage([toolSetRuntimeNode]), + nodes: [toolSetRuntimeNode], + storageNodes: encodeHttpToolSetNodesForStorage([toolSetRuntimeNode]), session }); diff --git a/projects/app/src/pages/api/core/app/httpTools/update.ts b/projects/app/src/pages/api/core/app/httpTools/update.ts index 596172330f1c..56e587f0b3ed 100644 --- a/projects/app/src/pages/api/core/app/httpTools/update.ts +++ b/projects/app/src/pages/api/core/app/httpTools/update.ts @@ -4,12 +4,11 @@ import { NextAPI } from '@/service/middleware/entry'; import type { ApiRequestProps } from '@fastgpt/next/type'; import { authApp } from '@fastgpt/service/support/permission/app/auth'; import { ManagePermissionVal } from '@fastgpt/global/support/permission/constant'; -import { MongoApp } from '@fastgpt/service/core/app/schema'; import { storeSecretValue } from '@fastgpt/service/common/secret/utils'; -import { MongoAppVersion } from '@fastgpt/service/core/app/version/schema'; import { updateParentFoldersUpdateTime } from '@fastgpt/service/core/app/controller'; import { beforeUpdateAppFormat } from '@fastgpt/service/core/app/controller'; import { parseApiInput } from '@fastgpt/service/common/zod/requestParseError'; +import { updateAppPublishedVersion } from '@fastgpt/service/core/app/version/controller'; import { UpdateHttpToolsBodySchema, UpdateHttpToolsResponseSchema, @@ -49,23 +48,12 @@ async function handler( await beforeUpdateAppFormat({ nodes: [toolSetRuntimeNode], teamId }); await mongoSessionRun(async (session) => { - await MongoApp.findByIdAndUpdate( + await updateAppPublishedVersion({ appId, - { - modules: storageNodes - }, - { session } - ); - - await MongoAppVersion.updateOne( - { appId }, - { - $set: { - nodes: storageNodes - } - }, - { session } - ); + nodes: storageNodes, + resources: [], + session + }); }); updateParentFoldersUpdateTime({ parentId: app.parentId diff --git a/projects/app/src/pages/api/core/app/list.ts b/projects/app/src/pages/api/core/app/list.ts index 69dc190248e8..52b93f41772c 100644 --- a/projects/app/src/pages/api/core/app/list.ts +++ b/projects/app/src/pages/api/core/app/list.ts @@ -1,4 +1,5 @@ import { MongoApp } from '@fastgpt/service/core/app/schema'; +import { MongoAppVersion } from '@fastgpt/service/core/app/version/schema'; import { NextAPI } from '@/service/middleware/entry'; import { PerResourceTypeEnum, @@ -18,9 +19,10 @@ import { replaceRegChars } from '@fastgpt/global/common/string/tools'; import { getGroupsByTmbId } from '@fastgpt/service/support/permission/memberGroup/controllers'; import { getOrgIdSetWithParentByTmbId } from '@fastgpt/service/support/permission/org/controllers'; import { addSourceMember } from '@fastgpt/service/support/user/utils'; -import { FlowNodeTypeEnum } from '@fastgpt/global/core/workflow/node/constant'; +import { isInteractiveNodeType } from '@fastgpt/global/core/workflow/node/constant'; import { isPrivateResourceByCollaborators, sumPer } from '@fastgpt/global/support/permission/utils'; import { getResourcePermissionsByTeam } from '@fastgpt/service/support/permission/resourcePermissionService'; +import { Types } from '@fastgpt/service/common/mongo'; import { parseApiInput } from '@fastgpt/service/common/zod/requestParseError'; import { ListAppBodySchema, @@ -28,7 +30,6 @@ import { type ListAppBodyType, type ListAppResponseType } from '@fastgpt/global/openapi/core/app/common/api'; -import { Types } from '@fastgpt/service/common/mongo'; /* 获取 APP 列表权限 @@ -170,7 +171,7 @@ async function handler(req: ApiRequestProps): Promise): Promise { + const pointerIds = myApps + .map((app) => app.publishedVersionId) + .filter((id): id is NonNullable => !!id && Types.ObjectId.isValid(String(id))); + if (pointerIds.length === 0) return new Set(); + + const versions = await MongoAppVersion.find( + { _id: { $in: pointerIds } }, + { _id: 1, appId: 1, nodes: 1 } + ).lean(); + const versionById = new Map(versions.map((version) => [String(version._id), version])); + const ids = new Set(); + + for (const app of myApps) { + const version = app.publishedVersionId + ? versionById.get(String(app.publishedVersionId)) + : undefined; + if (!version || String(version.appId) !== String(app._id)) continue; + if ((version.nodes ?? []).some((node) => isInteractiveNodeType(node.flowNodeType))) { + ids.add(String(app._id)); + } + } + + return ids; + }; + + const interactiveAppIds = await getInteractiveAppIdSet(); + // Add app permission and filter apps by read permission const formatApps = myApps .map((app) => { @@ -208,10 +241,7 @@ async function handler(req: ApiRequestProps): Promise - [FlowNodeTypeEnum.formInput, FlowNodeTypeEnum.userSelect].includes(item.flowNodeType) - ); + const { publishedVersionId: _publishedVersionId, ...rest } = app; return { ...rest, @@ -221,7 +251,7 @@ async function handler(req: ApiRequestProps): Promise app.permission.hasReadPer); diff --git a/projects/app/src/pages/api/core/app/mcpTools/create.ts b/projects/app/src/pages/api/core/app/mcpTools/create.ts index 51b5658ed4e0..339a7167b27a 100644 --- a/projects/app/src/pages/api/core/app/mcpTools/create.ts +++ b/projects/app/src/pages/api/core/app/mcpTools/create.ts @@ -63,8 +63,8 @@ async function handler( teamId, tmbId, type: AppTypeEnum.mcpToolSet, - modules: [toolSetRuntimeNode], - storageModules: encodeMcpToolSetNodesForStorage([toolSetRuntimeNode]), + nodes: [toolSetRuntimeNode], + storageNodes: encodeMcpToolSetNodesForStorage([toolSetRuntimeNode]), session }); diff --git a/projects/app/src/pages/api/core/app/mcpTools/update.ts b/projects/app/src/pages/api/core/app/mcpTools/update.ts index 8719adbb2df2..e75bec1df888 100644 --- a/projects/app/src/pages/api/core/app/mcpTools/update.ts +++ b/projects/app/src/pages/api/core/app/mcpTools/update.ts @@ -3,14 +3,13 @@ import { NextAPI } from '@/service/middleware/entry'; import { authApp } from '@fastgpt/service/support/permission/app/auth'; import { ManagePermissionVal } from '@fastgpt/global/support/permission/constant'; import { mongoSessionRun } from '@fastgpt/service/common/mongo/sessionRun'; -import { MongoApp } from '@fastgpt/service/core/app/schema'; import { getMCPToolSetRuntimeNode } from '@fastgpt/global/core/app/tool/mcpTool/utils'; -import { MongoAppVersion } from '@fastgpt/service/core/app/version/schema'; import { storeSecretValue } from '@fastgpt/service/common/secret/utils'; import { updateParentFoldersUpdateTime } from '@fastgpt/service/core/app/controller'; import { beforeUpdateAppFormat } from '@fastgpt/service/core/app/controller'; import { parseApiInput } from '@fastgpt/service/common/zod/requestParseError'; +import { updateAppPublishedVersion } from '@fastgpt/service/core/app/version/controller'; import { UpdateMcpToolsBodySchema, UpdateMcpToolsResponseSchema, @@ -48,25 +47,12 @@ async function handler( await beforeUpdateAppFormat({ nodes: [toolSetRuntimeNode], teamId }); await mongoSessionRun(async (session) => { - // update app and app version - await MongoApp.updateOne( - { _id: appId }, - { - modules: storageNodes, - updateTime: new Date() - }, - { session } - ); - - await MongoAppVersion.updateOne( - { appId }, - { - $set: { - nodes: storageNodes - } - }, - { session } - ); + await updateAppPublishedVersion({ + appId, + nodes: storageNodes, + resources: [], + session + }); }); updateParentFoldersUpdateTime({ parentId: app.parentId diff --git a/projects/app/src/pages/api/core/app/transitionWorkflow.ts b/projects/app/src/pages/api/core/app/transitionWorkflow.ts index 0bac81c97df0..4bdf23689943 100644 --- a/projects/app/src/pages/api/core/app/transitionWorkflow.ts +++ b/projects/app/src/pages/api/core/app/transitionWorkflow.ts @@ -14,7 +14,7 @@ import { type TransitionWorkflowBodyType, type TransitionWorkflowResponseType } from '@fastgpt/global/openapi/core/app/common/api'; -import { decodeToolSetNodesFromStorage } from '@fastgpt/service/core/app/jsonSchemaStorage'; +import { getAppDraftWorkflow } from '@fastgpt/service/core/app/version/controller'; async function handler( req: ApiRequestProps @@ -30,6 +30,7 @@ async function handler( authToken: true, per: OwnerPermissionVal }); + const draftWorkflow = await getAppDraftWorkflow(app._id); if (createNew) { const { appId } = await mongoSessionRun(async (session) => { // Copy avatar @@ -45,9 +46,9 @@ async function handler( name: app.name + ' Copy', avatar, type: AppTypeEnum.workflow, - modules: decodeToolSetNodesFromStorage(app.modules), - edges: app.edges, - chatConfig: app.chatConfig, + nodes: draftWorkflow.nodes, + edges: draftWorkflow.edges, + chatConfig: draftWorkflow.chatConfig, teamId: app.teamId, tmbId, session @@ -65,10 +66,12 @@ async function handler( await mongoSessionRun(async (session) => { await onUpdateAppWorkflow({ appId, - modules: decodeToolSetNodesFromStorage(app.modules), - edges: app.edges, - chatConfig: app.chatConfig, + nodes: draftWorkflow.nodes, + edges: draftWorkflow.edges, + chatConfig: draftWorkflow.chatConfig, + resources: draftWorkflow.resources, teamId, + tmbId, session }); }); diff --git a/projects/app/src/pages/api/core/app/version/detail.ts b/projects/app/src/pages/api/core/app/version/detail.ts index 079ab1680aa7..89e6f41c24b4 100644 --- a/projects/app/src/pages/api/core/app/version/detail.ts +++ b/projects/app/src/pages/api/core/app/version/detail.ts @@ -8,6 +8,7 @@ import { rewriteAppWorkflowToDetail } from '@fastgpt/service/core/app/utils'; import { migrateWorkflowToCurrent } from '@fastgpt/global/core/workflow/migration'; import { parseApiInput } from '@fastgpt/service/common/zod/requestParseError'; import { getLocale } from '@fastgpt/service/common/middle/i18n'; +import { resolveStoredAppResources } from '@fastgpt/service/core/app/resources'; import { GetAppVersionDetailQuerySchema, GetAppVersionDetailResponseSchema, @@ -21,7 +22,7 @@ async function handler(req: NextApiRequest): Promise) { nodes: normalizedWorkflow.nodes, teamId }); - if (isPublish) { - await validatePublishAppAgentSkillReadPermissions({ - nodes: normalizedWorkflow.nodes, - tmbId, - isRoot - }); - } - const resourceRefs = extractAppResourceRefsFromNodes(normalizedWorkflow.nodes); - updateParentFoldersUpdateTime({ - parentId: app.parentId + const extracted = extractAppResources({ + nodes: normalizedWorkflow.nodes, + chatConfig: normalizedWorkflow.chatConfig }); - if (autoSave) { await mongoSessionRun(async (session) => { - await MongoAppVersion.updateOne( + // baseline 必须在事务内读取,事务重试时才能按最新草稿重新计算资源快照。 + const resources = await resolveAppResourcesByPermission({ + appId, + extracted, + tmbId, + isRoot, + blockOnUnauthorized: false, + session + }); + await MongoAppVersion.findOneAndUpdate( { appId, isAutoSave: true @@ -78,29 +78,28 @@ async function handler(req: ApiRequestProps) { { tmbId, appId, + isAutoSave: true, nodes: normalizedWorkflow.nodes, edges: normalizedWorkflow.edges, chatConfig: normalizedWorkflow.chatConfig, versionName: i18nT('app:auto_save'), time: new Date(), - resourceRefs + resources }, - { session, upsert: true } + { session, upsert: true, new: true } ); - await MongoApp.updateOne( + const updateResult = await MongoApp.updateOne( { _id: appId }, - { - modules: normalizedWorkflow.nodes, - edges: normalizedWorkflow.edges, - chatConfig: normalizedWorkflow.chatConfig, - updateTime: new Date() - }, - { - session - } + { $set: { updateTime: new Date() } }, + { session } ); + if (updateResult.matchedCount !== 1) throw AppErrEnum.unExist; + }); + + updateParentFoldersUpdateTime({ + parentId: app.parentId }); addAuditLog({ @@ -119,6 +118,16 @@ async function handler(req: ApiRequestProps) { } await mongoSessionRun(async (session) => { + // baseline 必须在事务内读取,事务重试时才能按最新草稿重新计算资源快照。 + const resources = await resolveAppResourcesByPermission({ + appId, + extracted, + tmbId, + isRoot, + blockOnUnauthorized: !!isPublish, + session + }); + // create version histories const [{ _id }] = await MongoAppVersion.create( [ @@ -130,7 +139,7 @@ async function handler(req: ApiRequestProps) { isPublish, versionName, tmbId, - resourceRefs + resources } ], { session, ordered: true } @@ -138,12 +147,9 @@ async function handler(req: ApiRequestProps) { // update app const setUpdate = { - modules: normalizedWorkflow.nodes, - edges: normalizedWorkflow.edges, - chatConfig: normalizedWorkflow.chatConfig, updateTime: new Date(), version: 'v2', - ...(isPublish && { resourceRefs }), + ...(isPublish && { publishedVersionId: _id }), ...(isPublish && normalizedWorkflow.chatConfig.scheduledTriggerConfig?.cronString ? { scheduledTriggerConfig: normalizedWorkflow.chatConfig.scheduledTriggerConfig, @@ -154,7 +160,7 @@ async function handler(req: ApiRequestProps) { : {}), 'pluginData.nodeVersion': _id }; - await MongoApp.updateOne( + const updateResult = await MongoApp.updateOne( { _id: appId }, { $set: setUpdate, @@ -166,6 +172,11 @@ async function handler(req: ApiRequestProps) { session } ); + if (updateResult.matchedCount !== 1) throw AppErrEnum.unExist; + }); + + updateParentFoldersUpdateTime({ + parentId: app.parentId }); (async () => { diff --git a/projects/app/src/pages/api/core/chat/chatTest.ts b/projects/app/src/pages/api/core/chat/chatTest.ts index dbea0b3647e8..4fe3a5e56e64 100644 --- a/projects/app/src/pages/api/core/chat/chatTest.ts +++ b/projects/app/src/pages/api/core/chat/chatTest.ts @@ -59,6 +59,12 @@ import { import { buildChatSourceQuery } from '@fastgpt/service/core/chat/source'; import { APP_SANDBOX_ENABLED_CHAT_METADATA_KEY } from '@fastgpt/global/core/ai/sandbox/constants'; import { isAppSandboxEnabledInNodes } from '@fastgpt/global/core/workflow/utils'; +import { extractAppResources } from '@fastgpt/service/core/app/resources'; +import { resolveAppResourcesByPermission } from '@fastgpt/service/support/permission/app/resource'; +import { + getWorkflowResourceEntities, + loadWorkflowResourceContext +} from '@fastgpt/service/core/workflow/utils/resource'; async function handler(req: NextApiRequest, res: NextApiResponse) { let streamResponseContext: WorkflowStreamResponseContext | undefined; @@ -88,12 +94,30 @@ async function handler(req: NextApiRequest, res: NextApiResponse) { try { /* user auth */ - const { app, teamId, tmbId } = await authApp({ + const { app, teamId, tmbId, isRoot } = await authApp({ req, authToken: true, appId, per: ReadPermissionVal }); + const extractedResources = extractAppResources({ + nodes, + chatConfig + }); + const resourceContext = await loadWorkflowResourceContext({ + resources: extractedResources, + teamId, + isRoot + }); + await resolveAppResourcesByPermission({ + appId, + extracted: extractedResources, + tmbId, + isRoot, + blockOnUnauthorized: true, + allowRootCrossTeam: isRoot, + resourceEntities: getWorkflowResourceEntities(resourceContext) + }); // 类型获取 const isPlugin = app.type === AppTypeEnum.workflowTool; @@ -278,7 +302,8 @@ async function handler(req: NextApiRequest, res: NextApiResponse) { nodeResponseWriteConfig: { persistToDb: preparedRound.shouldPersistChatRound, retainInMemory: false - } + }, + resourceContext }); streamResponseContext.responseWrite(workflowSseEvent.answerStop()); diff --git a/projects/app/src/pages/api/core/workflow/debug.ts b/projects/app/src/pages/api/core/workflow/debug.ts index 7243d9e286b7..a7a1cf086736 100644 --- a/projects/app/src/pages/api/core/workflow/debug.ts +++ b/projects/app/src/pages/api/core/workflow/debug.ts @@ -8,7 +8,10 @@ import { getRunningUserInfoByTmbId } from '@fastgpt/service/support/user/team/ut import { NextAPI } from '@/service/middleware/entry'; import { ReadPermissionVal } from '@fastgpt/global/support/permission/constant'; import { WORKFLOW_MAX_RUN_TIMES } from '@fastgpt/service/core/workflow/constants'; -import { getLastInteractiveValue } from '@fastgpt/global/core/workflow/runtime/utils'; +import { + getLastInteractiveValue, + storeEdges2RuntimeEdges +} from '@fastgpt/global/core/workflow/runtime/utils'; import { getLocale } from '@fastgpt/service/common/middle/i18n'; import { createChatUsageRecord } from '@fastgpt/service/support/wallet/usage/controller'; import { getNanoid } from '@fastgpt/global/common/string/tools'; @@ -23,6 +26,13 @@ import { getWorkflowFinalResponseData } from '@/service/core/workflow/nodeResponse'; import { ChatSourceTypeEnum } from '@fastgpt/global/core/chat/constants'; +import { extractAppResources } from '@fastgpt/service/core/app/resources'; +import { resolveAppResourcesByPermission } from '@fastgpt/service/support/permission/app/resource'; +import { + getWorkflowResourceEntities, + loadWorkflowResourceContext +} from '@fastgpt/service/core/workflow/utils/resource'; +import { getAppDraftWorkflow } from '@fastgpt/service/core/app/version/controller'; async function handler(req: NextApiRequest, res: NextApiResponse): Promise { const { @@ -39,13 +49,15 @@ async function handler(req: NextApiRequest, res: NextApiResponse): Promise { if (isPlugin) { return serverGetWorkflowToolRunUserQuery({ - pluginInputs: getWorkflowToolInputsFromStoreNodes(app.modules), + pluginInputs: getWorkflowToolInputsFromStoreNodes(nodes), variables, files: variables.files }); @@ -234,12 +238,12 @@ async function handler(req: NextApiRequest, res: NextApiResponse) { })(); // Get and concat history; - const limit = getMaxHistoryLimitFromNodes(app.modules); + const limit = getMaxHistoryLimitFromNodes(nodes); const chatSource = { sourceType: ChatSourceTypeEnum.app, sourceId: String(app._id) }; - const [{ histories }, { versionId, nodes, edges, chatConfig }, chatDetail] = await Promise.all([ + const [{ histories }, chatDetail] = await Promise.all([ getChatItems({ ...chatSource, chatId, @@ -247,7 +251,6 @@ async function handler(req: NextApiRequest, res: NextApiResponse) { limit, field: `obj value memories nodeOutputs` }), - getAppLatestVersion(app._id, app), MongoChat.findOne( { ...buildChatSourceQuery(chatSource), chatId }, 'source variableList variables' @@ -392,6 +395,10 @@ async function handler(req: NextApiRequest, res: NextApiResponse) { maxBytesPerFile, lastInteractive: interactive, chatConfig, + resourceContext: await loadWorkflowResourceContext({ + resources, + teamId + }), histories: newHistories, stream, retainDatasetCite: workflowRetainDatasetCite, diff --git a/projects/app/src/pages/api/v2/chat/completions.ts b/projects/app/src/pages/api/v2/chat/completions.ts index 3702e7a15c6b..5f19bbb242ed 100644 --- a/projects/app/src/pages/api/v2/chat/completions.ts +++ b/projects/app/src/pages/api/v2/chat/completions.ts @@ -48,6 +48,7 @@ import { ChatErrEnum } from '@fastgpt/global/common/error/code/chat'; import { type AIChatItemType, type UserChatItemType } from '@fastgpt/global/core/chat/type'; import { NextAPI } from '@/service/middleware/entry'; import { getAppLatestVersion } from '@fastgpt/service/core/app/version/controller'; +import { loadWorkflowResourceContext } from '@fastgpt/service/core/workflow/utils/resource'; import { AppTypeEnum } from '@fastgpt/global/core/app/constants'; import { serverGetWorkflowToolRunUserQuery, @@ -215,11 +216,14 @@ async function handler(req: NextApiRequest, res: NextApiResponse) { } } + const workflowVersion = await getAppLatestVersion(app._id, app); + const { versionId, nodes, edges, chatConfig, resources } = workflowVersion; + // Get obj=Human history const userQuestion: UserChatItemType = (() => { if (isPlugin) { return serverGetWorkflowToolRunUserQuery({ - pluginInputs: getWorkflowToolInputsFromStoreNodes(app.modules), + pluginInputs: getWorkflowToolInputsFromStoreNodes(nodes), variables, files: variables.files }); @@ -233,12 +237,12 @@ async function handler(req: NextApiRequest, res: NextApiResponse) { })(); // Get and concat history; - const limit = getMaxHistoryLimitFromNodes(app.modules); + const limit = getMaxHistoryLimitFromNodes(nodes); const chatSource = { sourceType: ChatSourceTypeEnum.app, sourceId: String(app._id) }; - const [{ histories }, { versionId, nodes, edges, chatConfig }, chatDetail] = await Promise.all([ + const [{ histories }, chatDetail] = await Promise.all([ getChatItems({ ...chatSource, chatId, @@ -246,7 +250,6 @@ async function handler(req: NextApiRequest, res: NextApiResponse) { limit, field: `obj value memories nodeOutputs` }), - getAppLatestVersion(app._id, app), MongoChat.findOne( { ...buildChatSourceQuery({ @@ -395,6 +398,10 @@ async function handler(req: NextApiRequest, res: NextApiResponse) { maxBytesPerFile, lastInteractive: interactive, chatConfig, + resourceContext: await loadWorkflowResourceContext({ + resources, + teamId + }), histories: newHistories, stream, retainDatasetCite: workflowRetainDatasetCite, diff --git a/projects/app/src/pages/dashboard/create/index.tsx b/projects/app/src/pages/dashboard/create/index.tsx index d6c429b87e28..0fdf3dc71052 100644 --- a/projects/app/src/pages/dashboard/create/index.tsx +++ b/projects/app/src/pages/dashboard/create/index.tsx @@ -165,7 +165,7 @@ const CreateAppsPage = () => { avatar: templateDetail.avatar, name: templateDetail.name, type: appType, - modules: templateDetail.workflow.nodes || [], + nodes: templateDetail.workflow.nodes || [], edges: templateDetail.workflow.edges || [], chatConfig: templateDetail.workflow.chatConfig || {}, templateId: templateDetail.templateId @@ -176,7 +176,7 @@ const CreateAppsPage = () => { return postCreateApp({ ...baseParams, type: appType, - modules: emptyTemplate[appType].nodes, + nodes: emptyTemplate[appType].nodes, edges: emptyTemplate[appType].edges, chatConfig: emptyTemplate[appType].chatConfig }); diff --git a/projects/app/src/pages/dashboard/templateMarket/index.tsx b/projects/app/src/pages/dashboard/templateMarket/index.tsx index 61e8707c7384..612b23f7a0af 100644 --- a/projects/app/src/pages/dashboard/templateMarket/index.tsx +++ b/projects/app/src/pages/dashboard/templateMarket/index.tsx @@ -107,7 +107,7 @@ const TemplateMarket = ({ avatar: template.avatar, name: template.name, type: appType, - modules: templateDetail.workflow.nodes || [], + nodes: templateDetail.workflow.nodes || [], edges: templateDetail.workflow.edges || [], chatConfig: templateDetail.workflow.chatConfig, templateId: templateDetail.templateId diff --git a/projects/app/src/service/core/app/utils.ts b/projects/app/src/service/core/app/utils.ts index ce2aeeff810c..609e7788a7d5 100644 --- a/projects/app/src/service/core/app/utils.ts +++ b/projects/app/src/service/core/app/utils.ts @@ -35,6 +35,7 @@ import { dispatchWorkFlow } from '@fastgpt/service/core/workflow/dispatch'; import { prepareWorkflowFileQuery } from '@fastgpt/service/core/workflow/utils/fileLimits'; import { getRunningUserInfoByTmbId } from '@fastgpt/service/support/user/team/utils'; import { createChatUsageRecord } from '@fastgpt/service/support/wallet/usage/controller'; +import { loadWorkflowResourceContext } from '@fastgpt/service/core/workflow/utils/resource'; const logger = getLogger(); @@ -55,7 +56,8 @@ export const getScheduleTriggerApp = async () => { scheduledTriggerNextTime: 1, name: 1, teamId: 1, - tmbId: 1 + tmbId: 1, + publishedVersionId: 1 } ).lean(); }); @@ -71,9 +73,8 @@ export const getScheduleTriggerApp = async () => { let chatRoundFinalized = false; // Get app latest version - const { versionId, nodes, edges, chatConfig } = await retryFn(() => - getAppLatestVersion(app._id, app) - ); + const workflowVersion = await retryFn(() => getAppLatestVersion(app._id, app)); + const { versionId, nodes, edges, chatConfig, resources } = workflowVersion; const userQuery: UserChatItemValueItemType[] = [ { text: { @@ -190,6 +191,10 @@ export const getScheduleTriggerApp = async () => { uid: String(app.tmbId), runtimeNodes: storeNodes2RuntimeNodes(nodes, getWorkflowEntryNodeIds(nodes)), runtimeEdges: storeEdges2RuntimeEdges(edges), + resourceContext: await loadWorkflowResourceContext({ + resources, + teamId: app.teamId + }), variables: {}, query: workflowQuery, maxFileAmount, diff --git a/projects/app/src/service/core/app/workflow.ts b/projects/app/src/service/core/app/workflow.ts index 4cceef7e480a..b4cda39a9bde 100644 --- a/projects/app/src/service/core/app/workflow.ts +++ b/projects/app/src/service/core/app/workflow.ts @@ -3,6 +3,7 @@ import type { StoreNodeItemType } from '@fastgpt/global/core/workflow/type/node' import { isWorkflowSystemModelInput } from '@fastgpt/global/core/workflow/utils'; import { getOptionalLLMModelData } from '@fastgpt/service/core/ai/model'; +/** 从工作流主对话节点生成聊天页展示名称,不展示辅助模型资源。 */ export const getChatModelNameListByModules = (nodes: StoreNodeItemType[]): string[] => { const modelList = nodes .map((item) => { diff --git a/projects/app/src/service/core/sandbox/access.ts b/projects/app/src/service/core/sandbox/access.ts index d275b085db51..666b2b743f8b 100644 --- a/projects/app/src/service/core/sandbox/access.ts +++ b/projects/app/src/service/core/sandbox/access.ts @@ -8,7 +8,10 @@ import { type AppSandboxAvailability } from '@fastgpt/service/core/ai/sandbox/interface/runtime'; import { MongoApp } from '@fastgpt/service/core/app/schema'; -import { getAppLatestVersion } from '@fastgpt/service/core/app/version/controller'; +import { + getAppLatestVersion, + getAppDraftWorkflow +} from '@fastgpt/service/core/app/version/controller'; import { authSandboxSession, type AuthSandboxSessionParams, @@ -41,7 +44,7 @@ export async function resolveSandboxSessionAvailability( const app = await MongoApp.findById(session.sourceId).lean(); const appEnabled = isChatTest - ? isAppSandboxEnabledInNodes(app?.modules ?? []) + ? isAppSandboxEnabledInNodes((await getAppDraftWorkflow(session.sourceId)).nodes) : isAppSandboxEnabledInNodes( (await getAppLatestVersion(session.sourceId, app ?? undefined)).nodes ); diff --git a/projects/app/src/service/support/mcp/utils.ts b/projects/app/src/service/support/mcp/utils.ts index c8ac8bbc5c0d..189e77500905 100644 --- a/projects/app/src/service/support/mcp/utils.ts +++ b/projects/app/src/service/support/mcp/utils.ts @@ -49,6 +49,7 @@ import { authAppByTmbId } from '@fastgpt/service/support/permission/app/auth'; import { ReadPermissionVal } from '@fastgpt/global/support/permission/constant'; import { resolveMcpEffectiveTmbId } from './auth'; import { assertCancellation } from '@fastgpt/service/support/user/account/cancellation/guard'; +import { loadWorkflowResourceContext } from '@fastgpt/service/core/workflow/utils/resource'; const assertMcpTeamUsable = async (mcp: { teamId?: string; tmbId?: string }) => { if (!mcp.teamId || !mcp.tmbId) return; @@ -162,7 +163,7 @@ export const getMcpServerTools = async (key: string): Promise => { $in: [AppTypeEnum.simple, AppTypeEnum.workflow, AppTypeEnum.workflowTool] } }, - { name: 1, intro: 1 } + { name: 1, intro: 1, publishedVersionId: 1 } ).lean(); // Get latest version @@ -205,12 +206,15 @@ export const callMcpServerTool = async ({ key, toolName, inputs, authProxy }: to const pluginFixedTitle = isPlugin ? 'Mcp call' : undefined; // Get app latest version - const { versionId, nodes, edges, chatConfig } = await getAppLatestVersion(app._id, app); + const { versionId, nodes, edges, chatConfig, resources } = await getAppLatestVersion( + app._id, + app + ); const userQuestion: UserChatItemType = (() => { if (isPlugin) { return serverGetWorkflowToolRunUserQuery({ - pluginInputs: getWorkflowToolInputsFromStoreNodes(nodes || app.modules), + pluginInputs: getWorkflowToolInputsFromStoreNodes(nodes), variables }); } @@ -298,6 +302,10 @@ export const callMcpServerTool = async ({ key, toolName, inputs, authProxy }: to uid: effectiveTmbId, runtimeNodes, runtimeEdges: storeEdges2RuntimeEdges(edges), + resourceContext: await loadWorkflowResourceContext({ + resources, + teamId: app.teamId + }), variables, responseChatItemId: preparedRound.responseChatItemId, query: removeEmptyUserInput(workflowQuery), diff --git a/projects/app/src/web/core/app/constants.ts b/projects/app/src/web/core/app/constants.ts index a2e6a7eb1f06..91bd185521e0 100644 --- a/projects/app/src/web/core/app/constants.ts +++ b/projects/app/src/web/core/app/constants.ts @@ -13,7 +13,7 @@ export const defaultApp: AppDetailType = { avatar: '/icon/logo.svg', intro: '', updateTime: new Date(), - modules: [], + nodes: [], chatConfig: {}, teamId: '', tmbId: '', diff --git a/projects/app/src/web/core/app/templates.ts b/projects/app/src/web/core/app/templates.ts index d4f32b82305e..289b92d3b96c 100644 --- a/projects/app/src/web/core/app/templates.ts +++ b/projects/app/src/web/core/app/templates.ts @@ -1,7 +1,7 @@ import { getEmptyAgentConfig } from '@/pageComponents/app/detail/Edit/ChatAgent/utils'; import { parseCurl } from '@fastgpt/global/common/string/http'; import { AppTypeEnum } from '@fastgpt/global/core/app/constants'; -import { type AppSchemaType } from '@fastgpt/global/core/app/type'; +import { type AppVersionSchemaType } from '@fastgpt/global/core/app/version/type'; import { NodeInputKeyEnum, WorkflowIOValueTypeEnum } from '@fastgpt/global/core/workflow/constants'; import { FlowNodeInputTypeEnum, @@ -310,9 +310,9 @@ export const getEmptyAppsTemplate = (t: any) => { export const parsePluginFromCurlString = ( curl: string ): { - nodes: AppSchemaType['modules']; - edges: AppSchemaType['edges']; - chatConfig: AppSchemaType['chatConfig']; + nodes: AppVersionSchemaType['nodes']; + edges: AppVersionSchemaType['edges']; + chatConfig: AppVersionSchemaType['chatConfig']; } => { const { url, method, headers, body, params, bodyArray } = parseCurl(curl); diff --git a/projects/app/src/web/core/app/utils.ts b/projects/app/src/web/core/app/utils.ts index b25ea26bac1e..77a166b521b7 100644 --- a/projects/app/src/web/core/app/utils.ts +++ b/projects/app/src/web/core/app/utils.ts @@ -1,4 +1,3 @@ -import { type AppDetailType, type AppSchemaType } from '@fastgpt/global/core/app/type'; import type { AppFormEditFormType } from '@fastgpt/global/core/app/formEdit/type'; import { chatHistoryValueDesc } from '@fastgpt/global/core/workflow/node/constant'; import { NodeInputKeyEnum, WorkflowIOValueTypeEnum } from '@fastgpt/global/core/workflow/constants'; @@ -126,7 +125,3 @@ export const workflowSystemVariables: EditorVariablePickerType[] = [ valueType: WorkflowIOValueTypeEnum.string } ]; - -export const getAppQGuideCustomURL = (appDetail: AppDetailType | AppSchemaType): string => { - return appDetail.chatConfig?.chatInputGuide?.customUrl ?? ''; -}; diff --git a/projects/app/src/web/core/workflow/workflowCheck.ts b/projects/app/src/web/core/workflow/workflowCheck.ts index 68388c04828e..0f7a13bfd58d 100644 --- a/projects/app/src/web/core/workflow/workflowCheck.ts +++ b/projects/app/src/web/core/workflow/workflowCheck.ts @@ -171,7 +171,9 @@ type WorkflowCheckMessageCode = | 'tool_missing' | 'tool_load_failed' | 'tool_no_permission' - | 'model_unavailable'; + | 'model_unavailable' + | 'resource_missing' + | 'resource_no_permission'; /** issue.code -> 设计稿固定文案 code。表外 code 映射到最接近的已有文案。 */ const WORKFLOW_CHECK_ISSUE_MESSAGE_CODE_MAP: Record = { @@ -197,6 +199,8 @@ const WORKFLOW_CHECK_ISSUE_MESSAGE_CODE_MAP: Record([ 'tool_load_failed', 'tool_no_permission', 'tool_offline', - 'model_unavailable' + 'model_unavailable', + 'resource_missing', + 'resource_no_permission' ]); export type WorkflowCheckUIStatus = 'pending_improve' | 'pending_handle'; @@ -238,7 +244,9 @@ const workflowCheckMessageFallback: Record< tool_missing: () => '该工具不存在,请删除', tool_load_failed: () => '工具加载失败,请稍后重试', tool_no_permission: () => '当前账号无权限访问该资源', - model_unavailable: () => '模型已停用' + model_unavailable: () => '模型已停用', + resource_missing: () => '引用的知识库或技能已删除或不可用,请删除', + resource_no_permission: () => '无权限访问该资源,请检查权限' }; const PLUGIN_DATA_PERMISSION_ERROR_CODES = new Set([ @@ -321,6 +329,10 @@ const translateWorkflowCheckIssueMessage = ( return t('common:core.workflow.check.tool_no_permission', params); case 'model_unavailable': return t('common:core.workflow.check.model_unavailable', params); + case 'resource_missing': + return t('common:core.workflow.check.resource_missing', params); + case 'resource_no_permission': + return t('common:core.workflow.check.resource_no_permission', params); } }; @@ -589,6 +601,55 @@ export const checkWorkflowNodeIssues = ({ }); } + // 详情接口只会为当前操作者确认过的快照外资源设置 permissionDenied; + // 快照内历史资源不会重新鉴权,因此不会因协作者当前权限变化而误阻断。 + if (data.pluginData?.permissionDenied) { + addIssue({ + node, + code: 'resource_no_permission', + message: getWorkflowCheckIssueMessage('resource_no_permission', t) + }); + } + + // ACL 由保存/发布服务端按资源快照做差量校验;前端同步展示服务端返回的资源级标记, + // 并提示已删除或不可用的实体。 + const datasetParamsInput = inputMap.get(NodeInputKeyEnum.datasetParams); + const addResourceIssues = (value: unknown, inputKey: string) => { + if (!Array.isArray(value)) return; + if (value.some((item) => item && (item as { permissionDenied?: boolean }).permissionDenied)) { + addIssue({ + node, + code: 'resource_no_permission', + message: getWorkflowCheckIssueMessage('resource_no_permission', t), + inputKey + }); + } + if (value.some((item) => item && (item as { isDeleted?: boolean }).isDeleted)) { + addIssue({ + node, + code: 'resource_missing', + message: getWorkflowCheckIssueMessage('resource_missing', t), + inputKey + }); + } + }; + + [NodeInputKeyEnum.datasetSelectList, NodeInputKeyEnum.skills].forEach((key) => + addResourceIssues(inputMap.get(key)?.value, key) + ); + + if ( + data.flowNodeType === FlowNodeTypeEnum.agent && + datasetParamsInput?.value && + typeof datasetParamsInput.value === 'object' && + !Array.isArray(datasetParamsInput.value) + ) { + addResourceIssues( + (datasetParamsInput.value as { datasets?: unknown }).datasets, + NodeInputKeyEnum.datasetParams + ); + } + // 工具调用下游工具只有 systemInputConfig 未配置时才算未激活。 // 普通必填参数为空由下面的通用必填校验单独提示,不能复用整体工具配置状态。 const systemInputConfig = inputMap.get(NodeInputKeyEnum.systemInputConfig); @@ -642,7 +703,6 @@ export const checkWorkflowNodeIssues = ({ }); } - const datasetParamsInput = inputMap.get(NodeInputKeyEnum.datasetParams); if ( data.flowNodeType === FlowNodeTypeEnum.agent && datasetParamsInput?.value && diff --git a/projects/app/test/api/core/ai/skill/list.test.ts b/projects/app/test/api/core/ai/skill/list.test.ts index 6be4bfbb4146..6b00fde047b7 100644 --- a/projects/app/test/api/core/ai/skill/list.test.ts +++ b/projects/app/test/api/core/ai/skill/list.test.ts @@ -2,7 +2,6 @@ import { describe, expect, it } from 'vitest'; import handler from '@/pages/api/core/ai/skill/list'; import publishHandler from '@/pages/api/core/app/version/publish'; import { MongoAgentSkills } from '@fastgpt/service/core/ai/skill/model/schema'; -import { MongoApp } from '@fastgpt/service/core/app/schema'; import { MongoAppVersion } from '@fastgpt/service/core/app/version/schema'; import { AgentSkillSourceEnum, AgentSkillTypeEnum } from '@fastgpt/global/core/ai/skill/constants'; import { getNanoid } from '@fastgpt/global/common/string/tools'; @@ -275,7 +274,7 @@ describe('POST /api/core/ai/skill/list', () => { expect(res.data.list.map((item) => String(item._id))).toEqual([String(inheritedSkill._id)]); }); - it('appCount 基于已发布版本的 resourceRefs,草稿保存不影响统计', async () => { + it('appCount 基于已发布版本的 resources,草稿保存不影响统计', async () => { const user = await getUser(`agent-skill-list-published-refs-${getNanoid(6)}`); const [publishedSkill, draftSkill] = await MongoAgentSkills.create([ @@ -320,15 +319,12 @@ describe('POST /api/core/ai/skill/list', () => { const appId = await onCreateApp({ name: 'Skill Ref App', type: AppTypeEnum.chatAgent, - modules: [createSkillNode(publishedSkill)], + nodes: [createSkillNode(publishedSkill)], edges: [], chatConfig: {}, teamId: user.teamId, tmbId: user.tmbId }); - await expect(MongoApp.findById(appId).lean()).resolves.toMatchObject({ - resourceRefs: { skillIds: [String(publishedSkill._id)] } - }); const legacyModel = global.systemActiveModelList.find( (model) => model.type === ModelTypeEnum.llm @@ -354,11 +350,8 @@ describe('POST /api/core/ai/skill/list', () => { } }); expect(draftSaveRes.code).toBe(200); - const draftApp = await MongoApp.findById(appId).lean(); - expect(draftApp).toMatchObject({ - resourceRefs: { skillIds: [String(publishedSkill._id)] } - }); - expect(draftApp?.modules[0].inputs).toEqual( + const draftVersion = await MongoAppVersion.findOne({ appId, versionName: 'draft' }).lean(); + expect(draftVersion?.nodes[0].inputs).toEqual( expect.arrayContaining([ expect.objectContaining({ key: NodeInputKeyEnum.aiModelId, @@ -367,7 +360,7 @@ describe('POST /api/core/ai/skill/list', () => { ]) ); expect( - draftApp?.modules[0].inputs.some((input) => input.key === NodeInputKeyEnum.aiModel) + draftVersion?.nodes[0].inputs.some((input) => input.key === NodeInputKeyEnum.aiModel) ).toBe(false); const invalidDraftNode = createSkillNode(draftSkill); @@ -399,8 +392,11 @@ describe('POST /api/core/ai/skill/list', () => { } }); expect(invalidDraftSaveRes.code).toBe(200); - const invalidDraftApp = await MongoApp.findById(appId).lean(); - expect(invalidDraftApp?.modules[0].inputs).toEqual( + const invalidDraftVersion = await MongoAppVersion.findOne({ + appId, + versionName: 'invalid model draft' + }).lean(); + expect(invalidDraftVersion?.nodes[0].inputs).toEqual( expect.arrayContaining([ expect.objectContaining({ key: NodeInputKeyEnum.aiModelId, @@ -409,7 +405,7 @@ describe('POST /api/core/ai/skill/list', () => { ]) ); expect( - invalidDraftApp?.modules[0].inputs.some((input) => input.key === NodeInputKeyEnum.aiModel) + invalidDraftVersion?.nodes[0].inputs.some((input) => input.key === NodeInputKeyEnum.aiModel) ).toBe(false); const invalidPublishRes = await Call(publishHandler, { @@ -454,9 +450,6 @@ describe('POST /api/core/ai/skill/list', () => { } }); expect(publishRes.code).toBe(200); - await expect(MongoApp.findById(appId).lean()).resolves.toMatchObject({ - resourceRefs: { skillIds: [String(draftSkill._id)] } - }); const publishedRes = await Call, ListSkillsResponse>( handler, @@ -473,29 +466,5 @@ describe('POST /api/core/ai/skill/list', () => { publishedRes.data.list.find((item) => String(item._id) === skillId)?.appCount; expect(getPublishedCount(String(publishedSkill._id))).toBe(0); expect(getPublishedCount(String(draftSkill._id))).toBe(1); - - const latestPublishedVersion = await MongoAppVersion.findOne({ - appId, - isPublish: true - }) - .sort({ time: -1, _id: -1 }) - .lean(); - expect(latestPublishedVersion?.resourceRefs?.skillIds).toEqual([String(draftSkill._id)]); - await MongoApp.updateOne({ _id: appId }, { $set: { resourceRefs: { skillIds: [] } } }); - const clearedAppRefsRes = await Call< - ListSkillsQuery, - Record, - ListSkillsResponse - >(handler, { - auth: user, - body: { - source: 'mine', - parentId: null - } - }); - const getClearedAppRefsCount = (skillId: string) => - clearedAppRefsRes.data.list.find((item) => String(item._id) === skillId)?.appCount; - expect(getClearedAppRefsCount(String(publishedSkill._id))).toBe(0); - expect(getClearedAppRefsCount(String(draftSkill._id))).toBe(0); }); }); diff --git a/projects/app/test/api/core/app/copy.test.ts b/projects/app/test/api/core/app/copy.test.ts index 3ba1bc206b4e..82bc8fc19070 100644 --- a/projects/app/test/api/core/app/copy.test.ts +++ b/projects/app/test/api/core/app/copy.test.ts @@ -18,6 +18,7 @@ import { WritePermissionVal } from '@fastgpt/global/support/permission/constant' import { TeamAppCreatePermissionVal } from '@fastgpt/global/support/permission/user/constant'; import { mongoSessionRun } from '@fastgpt/service/common/mongo/sessionRun'; import { MongoApp } from '@fastgpt/service/core/app/schema'; +import { MongoAppVersion } from '@fastgpt/service/core/app/version/schema'; import { getResourceOwnedClbs } from '@fastgpt/service/support/permission/controller'; import { updateResourceCollaborators } from '@fastgpt/service/support/permission/resourcePermissionService'; import { MongoResourcePermission } from '@fastgpt/service/support/permission/schema'; @@ -41,7 +42,7 @@ describe('Copy', () => { { auth: users.members[0], body: { - modules: [], + nodes: [], name: 'testfolder', type: AppTypeEnum.folder } @@ -56,7 +57,7 @@ describe('Copy', () => { { auth: users.members[0], body: { - modules: [], + nodes: [], parentId: folderId, name: 'simple app', type: AppTypeEnum.simple @@ -66,24 +67,29 @@ describe('Copy', () => { expect(res2.error).toBeUndefined(); expect(res2.code).toBe(200); const appId = res2.data as string; - await MongoApp.findByIdAndUpdate(appId, { - modules: [ - { - nodeId: 'chat-node', - flowNodeType: FlowNodeTypeEnum.chatNode, - name: 'Chat', - inputs: [ + await MongoAppVersion.updateOne( + { appId, isPublish: true }, + { + $set: { + nodes: [ { - key: NodeInputKeyEnum.aiModel, - value: 'disabled-copy-model', - selectedType: FlowNodeInputTypeEnum.selectLLMModel, - renderTypeList: [FlowNodeInputTypeEnum.selectLLMModel] + nodeId: 'chat-node', + flowNodeType: FlowNodeTypeEnum.chatNode, + name: 'Chat', + inputs: [ + { + key: NodeInputKeyEnum.aiModel, + value: 'disabled-copy-model', + selectedType: FlowNodeInputTypeEnum.selectLLMModel, + renderTypeList: [FlowNodeInputTypeEnum.selectLLMModel] + } + ], + outputs: [] } - ], - outputs: [] + ] } - ] - }); + } + ); const res3 = await Call, CopyAppResponseType>( copyapi.default, @@ -132,11 +138,14 @@ describe('Copy', () => { ); expect(res4.error).toBeUndefined(); expect(res4.code).toBe(200); - const copiedApp = await MongoApp.findById(res4.data?.appId).lean(); + const copiedVersion = await MongoAppVersion.findOne({ + appId: res4.data?.appId, + isPublish: true + }).lean(); const expectedFallbackModelId = global.systemDefaultModel.llm?.modelId ?? global.systemActiveModelList.find((model) => model.type === ModelTypeEnum.llm)?.modelId; - expect(copiedApp?.modules[0].inputs).toEqual([ + expect(copiedVersion?.nodes[0].inputs).toEqual([ expect.objectContaining({ key: NodeInputKeyEnum.aiModelId, value: expectedFallbackModelId diff --git a/projects/app/test/api/core/app/create.test.ts b/projects/app/test/api/core/app/create.test.ts index 63ea51add2f9..8672f8b3f87d 100644 --- a/projects/app/test/api/core/app/create.test.ts +++ b/projects/app/test/api/core/app/create.test.ts @@ -1,12 +1,13 @@ import * as createapi from '@/pages/api/core/app/create'; import { AppErrEnum } from '@fastgpt/global/common/error/code/app'; import { AppTypeEnum } from '@fastgpt/global/core/app/constants'; -import type { AppSchemaType } from '@fastgpt/global/core/app/type'; +import type { AppVersionSchemaType } from '@fastgpt/global/core/app/version/type'; import { AppToolSourceEnum } from '@fastgpt/global/core/app/tool/constants'; import { FlowNodeInputTypeEnum } from '@fastgpt/global/core/workflow/node/constant'; import type { CreateAppBodyType } from '@fastgpt/global/openapi/core/app/common/api'; import { WritePermissionVal } from '@fastgpt/global/support/permission/constant'; import { TeamAppCreatePermissionVal } from '@fastgpt/global/support/permission/user/constant'; +import { Types } from '@fastgpt/service/common/mongo'; import { MongoApp } from '@fastgpt/service/core/app/schema'; import { MongoAppVersion } from '@fastgpt/service/core/app/version/schema'; import { MongoAppTemplate } from '@fastgpt/service/core/app/templates/templateSchema'; @@ -36,7 +37,7 @@ describe('create api', () => { const res = await Call(createapi.default, { auth: users.members[0], body: { - modules: [], + nodes: [], name: 'testfolder', type: AppTypeEnum.folder } @@ -48,7 +49,7 @@ describe('create api', () => { const res2 = await Call(createapi.default, { auth: users.members[0], body: { - modules: [], + nodes: [], name: 'testapp', type: AppTypeEnum.simple, parentId: String(folderId) @@ -61,7 +62,7 @@ describe('create api', () => { const res3 = await Call(createapi.default, { auth: users.members[1], body: { - modules: [], + nodes: [], name: 'testapp', type: AppTypeEnum.simple, parentId: String(folderId) @@ -86,7 +87,7 @@ describe('create api', () => { const res4 = await Call(createapi.default, { auth: users.members[1], body: { - modules: [], + nodes: [], name: 'testapp', type: AppTypeEnum.simple, parentId: String(folderId) @@ -121,7 +122,7 @@ describe('create api', () => { const res = await Call(createapi.default, { auth: users.members[0], body: { - modules: [], + nodes: [], name: 'community template app', avatar: '/plugin-avatar.png', type: AppTypeEnum.workflow, @@ -142,7 +143,7 @@ describe('create api', () => { type: AppTypeEnum.workflow, teamId: user.teamId, tmbId: user.tmbId, - modules: [ + nodes: [ { nodeId: 'start-1', flowNodeType: 'workflowStart', @@ -157,17 +158,20 @@ describe('create api', () => { ], outputs: [] } - ] as unknown as AppSchemaType['modules'] + ] as unknown as AppVersionSchemaType['nodes'] }); - const [app, version] = await Promise.all([ - MongoApp.findById(appId).lean(), - MongoAppVersion.findOne({ appId }).lean() - ]); - const input = app?.modules[0]?.inputs[0]; + const version = await MongoAppVersion.findOne({ appId }).lean(); + const rawApp = await MongoApp.collection.findOne( + { _id: new Types.ObjectId(appId) }, + { projection: { modules: 1, edges: 1, chatConfig: 1 } } + ); + const input = version?.nodes[0]?.inputs[0]; expect(input?.selectedType).toBe(FlowNodeInputTypeEnum.reference); expect(input).not.toHaveProperty('selectedTypeIndex'); - expect(version?.nodes).toEqual(app?.modules); + expect(rawApp).not.toHaveProperty('modules'); + expect(rawApp).not.toHaveProperty('edges'); + expect(rawApp).not.toHaveProperty('chatConfig'); }); }); diff --git a/projects/app/test/api/core/app/delete.test.ts b/projects/app/test/api/core/app/delete.test.ts index c9b6434db4f8..d38acf2bc0d3 100644 --- a/projects/app/test/api/core/app/delete.test.ts +++ b/projects/app/test/api/core/app/delete.test.ts @@ -20,7 +20,6 @@ import { MongoChatSetting } from '@fastgpt/service/core/chat/setting/schema'; import { MongoResourcePermission } from '@fastgpt/service/support/permission/schema'; import { PerResourceTypeEnum } from '@fastgpt/global/support/permission/constant'; import { MongoAppLogKeys } from '@fastgpt/service/core/app/logs/logkeysSchema'; -import { FlowNodeTypeEnum } from '@fastgpt/global/core/workflow/node/constant'; import { ChatSourceEnum, ChatSourceTypeEnum } from '@fastgpt/global/core/chat/constants'; import { MongoSystemTool } from '@fastgpt/service/core/plugin/tool/systemToolSchema'; @@ -155,8 +154,7 @@ describe('App Delete API Integration', () => { name: 'Test App for Deletion', teamId: rootUser.teamId, tmbId: rootUser.tmbId, - type: AppTypeEnum.simple, - modules: [] + type: AppTypeEnum.simple }); // Mock the queue to avoid actual background deletion @@ -211,8 +209,7 @@ describe('App Delete API Integration', () => { teamId: rootUser.teamId, tmbId: rootUser.tmbId, type: AppTypeEnum.simple, - parentId: testFolder._id, - modules: [] + parentId: testFolder._id }); // Mock the queue @@ -247,8 +244,7 @@ describe('App Delete API Integration', () => { name: 'Workflow Tool App for API Delete', teamId: rootUser.teamId, tmbId: rootUser.tmbId, - type: AppTypeEnum.workflowTool, - modules: [] + type: AppTypeEnum.workflowTool }); const pluginId = `commercial-api-delete-test-${Date.now()}`; @@ -310,18 +306,6 @@ describe('App Delete Data Cleanup Verification', () => { teamId: teamId, tmbId: rootUser.tmbId, type: AppTypeEnum.simple, - modules: [ - { - flowPosition: { x: 100, y: 100 }, - inputs: [], - outputs: [], - avatar: '/test/avatar.png', - name: 'Test Module', - intro: 'Test module intro', - flowType: FlowNodeTypeEnum.chatNode, - version: '1.0' - } - ], avatar: '/test/app-avatar.png' }); appId = String(testApp._id); @@ -365,8 +349,7 @@ describe('App Delete Data Cleanup Verification', () => { name: 'Parent App', teamId: teamId, tmbId: rootUser.tmbId, - type: AppTypeEnum.simple, - modules: [] + type: AppTypeEnum.simple }); const childApp = await MongoApp.create({ @@ -374,8 +357,7 @@ describe('App Delete Data Cleanup Verification', () => { teamId: teamId, tmbId: rootUser.tmbId, type: AppTypeEnum.simple, - parentId: parentApp._id, - modules: [] + parentId: parentApp._id }); // 为子应用创建相关数据 @@ -414,16 +396,14 @@ describe('App Delete Data Cleanup Verification', () => { name: 'Safety Parent App', teamId, tmbId: rootUser.tmbId, - type: AppTypeEnum.simple, - modules: [] + type: AppTypeEnum.simple }); const childApp = await MongoApp.create({ name: 'Safety Child App', teamId, tmbId: rootUser.tmbId, type: AppTypeEnum.simple, - parentId: parentApp._id, - modules: [] + parentId: parentApp._id }); await MongoApp.updateOne({ _id: parentApp._id }, { deleteTime: new Date() }); @@ -445,8 +425,7 @@ describe('App Delete Data Cleanup Verification', () => { name: 'Test App 2', teamId: teamId, tmbId: rootUser.tmbId, - type: AppTypeEnum.simple, - modules: [] + type: AppTypeEnum.simple }); const app2Id = String(app2._id); @@ -480,8 +459,7 @@ describe('App Delete Data Cleanup Verification', () => { name: 'Workflow Tool App', teamId, tmbId: rootUser.tmbId, - type: AppTypeEnum.workflowTool, - modules: [] + type: AppTypeEnum.workflowTool }); const workflowToolAppId = String(workflowToolApp._id); const pluginId = `commercial-delete-test-${Date.now()}`; diff --git a/projects/app/test/api/core/app/detail.test.ts b/projects/app/test/api/core/app/detail.test.ts new file mode 100644 index 000000000000..2f1a170a621e --- /dev/null +++ b/projects/app/test/api/core/app/detail.test.ts @@ -0,0 +1,54 @@ +import handler from '@/pages/api/core/app/detail'; +import { onCreateApp } from '@/pages/api/core/app/create'; +import { AppTypeEnum } from '@fastgpt/global/core/app/constants'; +import type { + GetAppDetailQueryType, + GetAppDetailResponseType +} from '@fastgpt/global/openapi/core/app/common/api'; +import { FlowNodeInputTypeEnum } from '@fastgpt/global/core/workflow/node/constant'; +import { getFakeUsers } from '@test/datas/users'; +import { Call } from '@test/utils/request'; +import { describe, expect, it } from 'vitest'; + +describe('get app detail api', () => { + it('returns draft workflow as nodes', async () => { + const [owner] = (await getFakeUsers(1)).members; + const appId = await onCreateApp({ + name: 'detail nodes app', + intro: '', + type: AppTypeEnum.workflow, + teamId: owner.teamId, + tmbId: owner.tmbId, + nodes: [ + { + nodeId: 'start-1', + flowNodeType: 'workflowStart', + name: 'Start', + inputs: [ + { + key: 'query', + label: 'Query', + renderTypeList: [FlowNodeInputTypeEnum.input] + } + ], + outputs: [] + } + ], + edges: [], + chatConfig: { welcomeText: 'hello' } + }); + + const res = await Call, GetAppDetailQueryType, GetAppDetailResponseType>( + handler, + { + auth: owner, + headers: {}, + query: { appId } + } + ); + + expect(res.code).toBe(200); + expect(res.data.nodes.map((node) => node.nodeId)).toEqual(['start-1']); + expect(res.data).not.toHaveProperty('modules'); + }); +}); diff --git a/projects/app/test/api/core/app/getPermission.test.ts b/projects/app/test/api/core/app/getPermission.test.ts index aedcbfaba9df..22d305faa371 100644 --- a/projects/app/test/api/core/app/getPermission.test.ts +++ b/projects/app/test/api/core/app/getPermission.test.ts @@ -17,8 +17,6 @@ describe('get app permission api', () => { const app = await MongoApp.create({ name: 'test-app', type: AppTypeEnum.simple, - modules: [], - edges: [], teamId: user.teamId, tmbId: user.tmbId }); @@ -69,8 +67,6 @@ describe('get app permission api', () => { const app = await MongoApp.create({ name: 'test-app', type: AppTypeEnum.simple, - modules: [], - edges: [], teamId: user1.teamId, tmbId: user1.tmbId }); diff --git a/projects/app/test/api/core/app/list.test.ts b/projects/app/test/api/core/app/list.test.ts index 027714f2b698..8475a809c990 100644 --- a/projects/app/test/api/core/app/list.test.ts +++ b/projects/app/test/api/core/app/list.test.ts @@ -1,6 +1,8 @@ import handler from '@/pages/api/core/app/list'; import { FlowNodeTypeEnum } from '@fastgpt/global/core/workflow/node/constant'; import { AppListSortEnum, AppTypeEnum } from '@fastgpt/global/core/app/constants'; +import { onCreateApp } from '@/pages/api/core/app/create'; +import { getNanoid } from '@fastgpt/global/common/string/tools'; import { ReadPermissionVal, PerResourceTypeEnum @@ -12,51 +14,60 @@ import type { import { Types } from '@fastgpt/service/common/mongo'; import { MongoApp } from '@fastgpt/service/core/app/schema'; import { MongoResourcePermission } from '@fastgpt/service/support/permission/schema'; -import { getFakeUsers } from '@test/datas/users'; +import { getFakeUsers, getUser } from '@test/datas/users'; import { Call } from '@test/utils/request'; import { describe, expect, it } from 'vitest'; -describe('POST /api/core/app/list', () => { - it('derives the interactive-node flag from the projected flow node type', async () => { - const { owner } = await getFakeUsers(1); - await MongoApp.create([ - { - name: 'Interactive app', - type: AppTypeEnum.workflow, - teamId: owner.teamId, - tmbId: owner.tmbId, - modules: [ - { - flowNodeType: FlowNodeTypeEnum.formInput, - inputs: [{ key: 'large-input', value: 'content that the list response does not need' }] - } - ] - }, - { - name: 'Regular app', - type: AppTypeEnum.workflow, - teamId: owner.teamId, - tmbId: owner.tmbId, - modules: [{ flowNodeType: FlowNodeTypeEnum.chatNode, inputs: [] }] - } - ]); +const startNode = { + nodeId: 'start-1', + flowNodeType: FlowNodeTypeEnum.workflowStart, + name: 'Start', + inputs: [], + outputs: [] +}; - const response = await Call, ListAppResponseType>( - handler, - { - auth: owner, - body: { type: AppTypeEnum.workflow } - } - ); +const formInputNode = { + nodeId: 'form-1', + flowNodeType: FlowNodeTypeEnum.formInput, + name: 'Form', + inputs: [], + outputs: [] +}; - expect(response.code).toBe(200); - expect(response.data).toEqual( - expect.arrayContaining([ - expect.objectContaining({ name: 'Interactive app', hasInteractiveNode: true }), - expect.objectContaining({ name: 'Regular app', hasInteractiveNode: false }) - ]) +describe('POST /api/core/app/list', () => { + it('reads hasInteractiveNode from published version nodes', async () => { + const owner = await getUser(`app-list-interactive-${getNanoid(6)}`); + const interactiveAppId = await onCreateApp({ + name: 'interactive app', + intro: '', + type: AppTypeEnum.workflow, + teamId: owner.teamId, + tmbId: owner.tmbId, + nodes: [startNode, formInputNode] + }); + const leftoverModulesAppId = await onCreateApp({ + name: 'leftover modules app', + intro: '', + type: AppTypeEnum.workflow, + teamId: owner.teamId, + tmbId: owner.tmbId, + nodes: [startNode] + }); + await MongoApp.collection.updateOne( + { _id: new Types.ObjectId(leftoverModulesAppId) }, + { $set: { modules: [formInputNode] } } ); - expect(response.data.every((app) => !('modules' in app))).toBe(true); + + const res = await Call, ListAppResponseType>(handler, { + auth: owner, + body: {} + }); + const findApp = (appId: string) => res.data.find((item) => String(item._id) === appId); + + expect(res.code).toBe(200); + expect(findApp(interactiveAppId)?.hasInteractiveNode).toBe(true); + expect(findApp(leftoverModulesAppId)?.hasInteractiveNode).toBe(false); + expect(res.data.every((app) => !('modules' in app))).toBe(true); }); it('returns only apps covered by the current member resource permission group', async () => { @@ -66,15 +77,13 @@ describe('POST /api/core/app/list', () => { name: 'Permitted app', type: AppTypeEnum.workflow, teamId: owner.teamId, - tmbId: owner.tmbId, - modules: [] + tmbId: owner.tmbId }, { name: 'Inaccessible app', type: AppTypeEnum.workflow, teamId: owner.teamId, - tmbId: owner.tmbId, - modules: [] + tmbId: owner.tmbId } ]); await MongoResourcePermission.create([ @@ -194,4 +203,44 @@ describe('POST /api/core/app/list', () => { ); expect(emptied.data).toEqual([]); }); + + it('applies permission filtering before the search limit', async () => { + const { owner, members } = await getFakeUsers(2); + const [permittedApp] = await MongoApp.create([ + { + name: 'needle permitted app', + type: AppTypeEnum.workflow, + teamId: owner.teamId, + tmbId: owner.tmbId + } + ]); + await MongoApp.create( + Array.from({ length: 60 }, (_, index) => ({ + name: `needle inaccessible app ${String(index).padStart(2, '0')}`, + type: AppTypeEnum.workflow, + teamId: owner.teamId, + tmbId: owner.tmbId + })) + ); + await MongoResourcePermission.create({ + resourceType: PerResourceTypeEnum.app, + teamId: owner.teamId, + resourceId: permittedApp._id, + tmbId: members[0].tmbId, + permission: ReadPermissionVal + }); + + const response = await Call, ListAppResponseType>( + handler, + { + auth: members[0], + body: { searchKey: 'needle' } + } + ); + + expect(response.code).toBe(200); + expect(response.data).toEqual([ + expect.objectContaining({ _id: String(permittedApp._id), name: 'needle permitted app' }) + ]); + }); }); diff --git a/projects/app/test/api/core/app/toolSetUpdate.test.ts b/projects/app/test/api/core/app/toolSetUpdate.test.ts new file mode 100644 index 000000000000..cb2ebc1e4c07 --- /dev/null +++ b/projects/app/test/api/core/app/toolSetUpdate.test.ts @@ -0,0 +1,95 @@ +import { beforeEach, describe, expect, it, vi } from 'vitest'; +import type { ApiRequestProps } from '@fastgpt/next/type'; +import type { UpdateHttpToolsBodyType } from '@fastgpt/global/openapi/core/app/httpTools/api'; +import type { UpdateMcpToolsBodyType } from '@fastgpt/global/openapi/core/app/mcpTools/api'; +import { AppErrEnum } from '@fastgpt/global/common/error/code/app'; + +const mocks = vi.hoisted(() => ({ + authApp: vi.fn(), + mongoSessionRun: vi.fn(), + updateAppPublishedVersion: vi.fn(), + beforeUpdateAppFormat: vi.fn(), + updateParentFoldersUpdateTime: vi.fn(), + assertMCPUrlNotInternal: vi.fn() +})); + +vi.mock('@/service/middleware/entry', () => ({ + NextAPI: (handler: unknown) => handler +})); + +vi.mock('@fastgpt/service/support/permission/app/auth', () => ({ + authApp: mocks.authApp +})); + +vi.mock('@fastgpt/service/common/mongo/sessionRun', () => ({ + mongoSessionRun: mocks.mongoSessionRun +})); + +vi.mock('@fastgpt/service/core/app/version/controller', () => ({ + updateAppPublishedVersion: mocks.updateAppPublishedVersion +})); + +vi.mock('@fastgpt/service/core/app/controller', () => ({ + beforeUpdateAppFormat: mocks.beforeUpdateAppFormat, + updateParentFoldersUpdateTime: mocks.updateParentFoldersUpdateTime +})); + +vi.mock('@fastgpt/service/common/secret/utils', () => ({ + storeSecretValue: vi.fn((value) => value) +})); + +vi.mock('@fastgpt/service/core/app/mcp', () => ({ + assertMCPUrlNotInternal: mocks.assertMCPUrlNotInternal +})); + +import updateHttpTools from '@/pages/api/core/app/httpTools/update'; +import updateMcpTools from '@/pages/api/core/app/mcpTools/update'; + +const appId = '65f000000000000000000071'; +const versionId = '65f000000000000000000072'; + +describe('tool set update handlers', () => { + beforeEach(() => { + vi.clearAllMocks(); + mocks.authApp.mockResolvedValue({ + app: { + _id: appId, + name: 'Tool set', + avatar: '', + parentId: 'parent-id', + publishedVersionId: versionId + }, + teamId: 'team-id' + }); + mocks.beforeUpdateAppFormat.mockResolvedValue(undefined); + mocks.assertMCPUrlNotInternal.mockResolvedValue(undefined); + mocks.mongoSessionRun.mockImplementation(async (fn: (session: string) => Promise) => + fn('session') + ); + mocks.updateAppPublishedVersion.mockRejectedValue(AppErrEnum.unExist); + }); + + it.each([ + { + name: 'HTTP', + handler: updateHttpTools, + body: { appId, toolList: [] } as UpdateHttpToolsBodyType + }, + { + name: 'MCP', + handler: updateMcpTools, + body: { appId, url: 'https://example.com/mcp', toolList: [] } as UpdateMcpToolsBodyType + } + ])('propagates a $name published-version update failure', async ({ handler, body }) => { + await expect(handler({ body } as ApiRequestProps)).rejects.toBe(AppErrEnum.unExist); + + expect(mocks.updateAppPublishedVersion).toHaveBeenCalledWith( + expect.objectContaining({ + appId, + resources: [], + session: 'session' + }) + ); + expect(mocks.updateParentFoldersUpdateTime).not.toHaveBeenCalled(); + }); +}); diff --git a/projects/app/test/api/core/app/transitionWorkflow.test.ts b/projects/app/test/api/core/app/transitionWorkflow.test.ts index e26521d94e86..7dd8c725876d 100644 --- a/projects/app/test/api/core/app/transitionWorkflow.test.ts +++ b/projects/app/test/api/core/app/transitionWorkflow.test.ts @@ -49,10 +49,14 @@ describe('Transition workflow', () => { CreateAppResponseType >(createapi.default, { auth: user, - body: { name: 'simple app', type: AppTypeEnum.simple, modules: [] } + body: { name: 'simple app', type: AppTypeEnum.simple, nodes: [] } }); const sourceAppId = createResult.data!; - await MongoApp.updateOne({ _id: sourceAppId }, { modules: historicalModules }); + const copiedResources = [{ type: 'skill' as const, id: 'copied-skill' }]; + await MongoAppVersion.updateOne( + { appId: sourceAppId }, + { nodes: historicalModules, resources: copiedResources } + ); const result = await Call< TransitionWorkflowBodyType, @@ -65,16 +69,18 @@ describe('Transition workflow', () => { const appId = createNew ? result.data?.id : sourceAppId; const [app, version] = await Promise.all([ MongoApp.findById(appId).lean(), - createNew ? MongoAppVersion.findOne({ appId }).lean() : undefined + createNew + ? MongoAppVersion.findOne({ appId }).lean() + : MongoAppVersion.findOne({ appId, isAutoSave: true }).lean() ]); - const input = app?.modules[0]?.inputs[0]; + const input = version?.nodes[0]?.inputs[0]; expect(result.code).toBe(200); expect(app?.type).toBe(AppTypeEnum.workflow); expect(input?.selectedType).toBe(FlowNodeInputTypeEnum.reference); expect(input).not.toHaveProperty('selectedTypeIndex'); - if (createNew) { - expect(version?.nodes).toEqual(app?.modules); + if (!createNew) { + expect(version?.resources).toEqual(copiedResources); } }); }); diff --git a/projects/app/test/api/core/app/version/detail.test.ts b/projects/app/test/api/core/app/version/detail.test.ts new file mode 100644 index 000000000000..3737adbbaa8a --- /dev/null +++ b/projects/app/test/api/core/app/version/detail.test.ts @@ -0,0 +1,53 @@ +import { MongoApp } from '@fastgpt/service/core/app/schema'; +import { MongoAppVersion } from '@fastgpt/service/core/app/version/schema'; +import { getRootUser } from '@test/datas/users'; +import { Call } from '@test/utils/request'; +import { describe, expect, it } from 'vitest'; +import handler from '@/pages/api/core/app/version/detail'; +import type { + GetAppVersionDetailQueryType, + GetAppVersionDetailResponseType +} from '@fastgpt/global/openapi/core/app/version/api'; + +describe('app version detail API resources', () => { + it('extracts canonical model resources for a version without a stored snapshot', async () => { + const root = await getRootUser(); + const app = await MongoApp.create({ + name: 'legacy version detail', + tmbId: root.tmbId, + teamId: root.teamId + }); + const version = await MongoAppVersion.create({ + tmbId: root.tmbId, + appId: app._id, + nodes: [], + edges: [], + chatConfig: { + questionGuide: { + open: true, + modelId: 'guide-model-id' + } + }, + versionName: 'legacy' + }); + await MongoAppVersion.updateOne({ _id: version._id }, { $unset: { resources: 1 } }); + + const res = await Call< + Record, + GetAppVersionDetailQueryType, + GetAppVersionDetailResponseType + >(handler, { + auth: root, + headers: {}, + query: { + appId: String(app._id), + versionId: String(version._id) + } + }); + + expect(res.code).toBe(200); + expect(res.data.resources).toEqual([ + { type: 'model', id: 'guide-model-id', data: { modelType: 'llm' } } + ]); + }); +}); diff --git a/projects/app/test/api/core/workflow/debug.test.ts b/projects/app/test/api/core/workflow/debug.test.ts index 6574766f65f1..305ead876b07 100644 --- a/projects/app/test/api/core/workflow/debug.test.ts +++ b/projects/app/test/api/core/workflow/debug.test.ts @@ -7,6 +7,8 @@ const mocks = vi.hoisted(() => ({ createChatUsageRecord: vi.fn(), dispatchWorkFlow: vi.fn(), getNanoid: vi.fn(), + getAppDraftWorkflow: vi.fn(), + getAppDraftResourceBaseline: vi.fn(), getRunningUserInfoByTmbId: vi.fn(), getWorkflowFinalResponseData: vi.fn(), prepareWorkflowFileQuery: vi.fn() @@ -28,6 +30,11 @@ vi.mock('@fastgpt/service/core/workflow/dispatch', () => ({ dispatchWorkFlow: mocks.dispatchWorkFlow })); +vi.mock('@fastgpt/service/core/app/version/controller', () => ({ + getAppDraftWorkflow: mocks.getAppDraftWorkflow, + getAppDraftResourceBaseline: mocks.getAppDraftResourceBaseline +})); + vi.mock('@fastgpt/service/core/workflow/utils/fileLimits', () => ({ prepareWorkflowFileQuery: mocks.prepareWorkflowFileQuery })); @@ -61,15 +68,22 @@ const app = { _id: appId, name: 'Workflow app', teamId: 'team-id', - tmbId: 'owner-id', - chatConfig: { variables: [] } + tmbId: 'owner-id' }; +const savedChatConfig = { variables: [] }; describe('workflow debug API chatId', () => { beforeEach(() => { vi.clearAllMocks(); mocks.authCert.mockResolvedValue({ tmbId: 'member-id' }); mocks.authApp.mockResolvedValue({ app }); + mocks.getAppDraftWorkflow.mockResolvedValue({ + nodes: [], + edges: [], + chatConfig: savedChatConfig, + resources: [] + }); + mocks.getAppDraftResourceBaseline.mockResolvedValue([]); mocks.getRunningUserInfoByTmbId.mockResolvedValue({ teamId: 'team-id', tmbId: 'member-id' @@ -117,6 +131,10 @@ describe('workflow debug API chatId', () => { ); expect(mocks.createChatUsageRecord).not.toHaveBeenCalled(); expect(mocks.getNanoid).toHaveBeenCalledTimes(1); + expect(mocks.getAppDraftWorkflow).toHaveBeenCalledWith(appId); + expect(mocks.prepareWorkflowFileQuery).toHaveBeenCalledWith( + expect.objectContaining({ chatConfig: savedChatConfig }) + ); }); it('generates a dispatch chatId when an older client does not provide one', async () => { @@ -142,5 +160,30 @@ describe('workflow debug API chatId', () => { ); expect(mocks.createChatUsageRecord).toHaveBeenCalledTimes(1); expect(mocks.getNanoid).toHaveBeenCalledTimes(2); + expect(mocks.getAppDraftWorkflow).toHaveBeenCalledWith(appId); + expect(mocks.dispatchWorkFlow).toHaveBeenCalledWith( + expect.objectContaining({ chatConfig: savedChatConfig }) + ); + }); + + it('uses request chatConfig without loading the draft workflow', async () => { + const requestChatConfig = { variables: [] }; + mocks.getNanoid.mockReturnValue('response-chat-item-id'); + + await handler( + { + body: { appId, chatConfig: requestChatConfig, usageId: 'usage-id' }, + headers: {} + } as any, + {} as any + ); + + expect(mocks.getAppDraftWorkflow).not.toHaveBeenCalled(); + expect(mocks.prepareWorkflowFileQuery).toHaveBeenCalledWith( + expect.objectContaining({ chatConfig: requestChatConfig }) + ); + expect(mocks.dispatchWorkFlow).toHaveBeenCalledWith( + expect.objectContaining({ chatConfig: requestChatConfig }) + ); }); }); diff --git a/projects/app/test/pages/api/admin/4163/initAppResources.test.ts b/projects/app/test/pages/api/admin/4163/initAppResources.test.ts new file mode 100644 index 000000000000..8327e320d471 --- /dev/null +++ b/projects/app/test/pages/api/admin/4163/initAppResources.test.ts @@ -0,0 +1,355 @@ +import { beforeEach, describe, expect, it, vi } from 'vitest'; +import { Types } from '@fastgpt/service/common/mongo'; +import { MongoApp } from '@fastgpt/service/core/app/schema'; +import { MongoAppVersion } from '@fastgpt/service/core/app/version/schema'; + +vi.mock('@/service/middleware/entry', () => ({ + NextAPI: (handler: unknown) => handler +})); + +import { runInitAppResourcesMigration } from '@/pages/api/admin/4163/initAppResources'; + +const teamId = new Types.ObjectId('65f000000000000000000061'); +const tmbId = new Types.ObjectId('65f000000000000000000062'); +const appId = new Types.ObjectId('65f000000000000000000063'); +const versionId = new Types.ObjectId('65f000000000000000000064'); + +const createLegacyRecords = ({ + currentAppId = appId, + currentVersionId = versionId, + appModules = [], + versionNodes = [] +}: { + currentAppId?: Types.ObjectId; + currentVersionId?: Types.ObjectId; + appModules?: unknown[]; + versionNodes?: unknown[]; +} = {}) => ({ + app: { + _id: currentAppId, + teamId, + tmbId, + name: 'Legacy app', + type: 'workflow', + modules: appModules, + edges: [], + chatConfig: {}, + resourceRefs: { + skillIds: ['legacy-skill'] + } + }, + version: { + _id: currentVersionId, + appId: currentAppId, + tmbId, + time: new Date('2026-08-20T00:00:00.000Z'), + isPublish: true, + versionName: 'Published version', + nodes: versionNodes, + edges: [], + chatConfig: {}, + resourceRefs: { + skillIds: ['published-skill'] + } + } +}); + +const insertLegacyRecords = async (records = [createLegacyRecords()]) => { + await Promise.all([ + MongoApp.collection.insertMany(records.map(({ app }) => app)), + MongoAppVersion.collection.insertMany(records.map(({ version }) => version)) + ]); +}; + +const insertLegacyAppWithoutVersions = async (appModules: unknown[] = []) => { + await MongoApp.collection.insertOne(createLegacyRecords({ appModules }).app); +}; + +describe('initAppResources migration API', () => { + beforeEach(async () => { + vi.restoreAllMocks(); + vi.clearAllMocks(); + await Promise.all([MongoApp.deleteMany({}), MongoAppVersion.deleteMany({})]); + }); + + it('dry-runs without writing App or App Version resources', async () => { + await insertLegacyRecords(); + + const result = await runInitAppResourcesMigration({ dryRun: true }); + + expect(result).toMatchObject({ + dryRun: true, + stats: { + appsScanned: 1, + versionsScanned: 1, + appsUpdated: 0, + versionsUpdated: 0, + legacySkillRefs: 2 + } + }); + expect( + await MongoApp.collection.findOne({ _id: appId }, { projection: { resourceRefs: 1 } }) + ).toMatchObject({ resourceRefs: { skillIds: ['legacy-skill'] } }); + expect( + await MongoAppVersion.collection.findOne( + { _id: versionId }, + { projection: { resourceRefs: 1 } } + ) + ).toMatchObject({ resourceRefs: { skillIds: ['published-skill'] } }); + }); + + it('writes resources in batches and removes legacy resource references', async () => { + const records = Array.from({ length: 3 }, () => + createLegacyRecords({ + currentAppId: new Types.ObjectId(), + currentVersionId: new Types.ObjectId() + }) + ); + await insertLegacyRecords(records); + + const result = await runInitAppResourcesMigration({ + dryRun: false, + batchSize: 1, + writeBatchSize: 1 + }); + + expect(result).toMatchObject({ + dryRun: false, + batchSize: 1, + writeBatchSize: 1, + stats: { + appsScanned: 3, + versionsScanned: 3, + appsUpdated: 3, + versionsUpdated: 3 + } + }); + expect(await MongoApp.collection.findOne({ _id: records[0].app._id })).toMatchObject({ + publishedVersionId: records[0].version._id + }); + expect(await MongoApp.collection.findOne({ _id: records[0].app._id })).not.toHaveProperty( + 'modules' + ); + expect(await MongoAppVersion.collection.findOne({ _id: records[0].version._id })).toMatchObject( + { + resources: [{ type: 'skill', id: 'published-skill' }] + } + ); + expect(await MongoApp.countDocuments({ resourceRefs: { $exists: true } })).toBe(0); + expect(await MongoAppVersion.countDocuments({ resourceRefs: { $exists: true } })).toBe(0); + }); + + it('skips records changed after they were read and preserves their retry references', async () => { + await insertLegacyRecords(); + + const originalVersionBulkWrite = MongoAppVersion.collection.bulkWrite.bind( + MongoAppVersion.collection + ); + let hasConcurrentChange = false; + vi.spyOn(MongoAppVersion.collection, 'bulkWrite').mockImplementation( + async (operations, options) => { + if (!hasConcurrentChange) { + hasConcurrentChange = true; + await MongoAppVersion.collection.updateOne( + { _id: versionId }, + { $set: { nodes: [{ nodeId: 'changed-after-read' }] } } + ); + } + return originalVersionBulkWrite(operations, options); + } + ); + + const result = await runInitAppResourcesMigration({ + dryRun: false, + batchSize: 1, + writeBatchSize: 1 + }); + + expect(result).toMatchObject({ + stats: { + appsScanned: 1, + versionsScanned: 1, + appsUpdated: 0, + versionsUpdated: 0, + appsSkipped: 1, + versionsSkipped: 1 + } + }); + expect(await MongoAppVersion.collection.findOne({ _id: versionId })).toMatchObject({ + nodes: [{ nodeId: 'changed-after-read' }], + resourceRefs: { skillIds: ['published-skill'] } + }); + expect( + await MongoApp.collection.findOne( + { _id: appId }, + { projection: { resourceRefs: 1, publishedVersionId: 1 } } + ) + ).toMatchObject({ resourceRefs: { skillIds: ['legacy-skill'] } }); + expect(await MongoAppVersion.countDocuments({ resourceRefs: { $exists: true } })).toBe(1); + expect(await MongoApp.countDocuments({ resourceRefs: { $exists: true } })).toBe(1); + }); + + it('does not unset App graph after the App changes during migration', async () => { + await insertLegacyRecords(); + + const originalAppBulkWrite = MongoApp.collection.bulkWrite.bind(MongoApp.collection); + let hasConcurrentChange = false; + vi.spyOn(MongoApp.collection, 'bulkWrite').mockImplementation(async (operations, options) => { + if (!hasConcurrentChange) { + hasConcurrentChange = true; + await MongoApp.collection.updateOne( + { _id: appId }, + { $set: { modules: [{ nodeId: 'changed-after-read' }] } } + ); + } + return originalAppBulkWrite(operations, options); + }); + + const result = await runInitAppResourcesMigration({ + dryRun: false, + batchSize: 1, + writeBatchSize: 1 + }); + + expect(result).toMatchObject({ + stats: { + appsScanned: 1, + versionsScanned: 1, + appsUpdated: 0, + versionsUpdated: 1, + appsSkipped: 1, + versionsSkipped: 0 + } + }); + expect(await MongoApp.collection.findOne({ _id: appId })).toMatchObject({ + modules: [{ nodeId: 'changed-after-read' }], + resourceRefs: { skillIds: ['legacy-skill'] } + }); + expect(await MongoAppVersion.collection.findOne({ _id: versionId })).toMatchObject({ + resources: [{ type: 'skill', id: 'published-skill' }] + }); + expect(await MongoApp.countDocuments({ resourceRefs: { $exists: true } })).toBe(1); + expect(await MongoAppVersion.countDocuments({ resourceRefs: { $exists: true } })).toBe(0); + }); + + it('does not overwrite a concurrently published version pointer', async () => { + await insertLegacyRecords(); + const concurrentVersionId = new Types.ObjectId(); + + const originalAppBulkWrite = MongoApp.collection.bulkWrite.bind(MongoApp.collection); + let hasConcurrentChange = false; + vi.spyOn(MongoApp.collection, 'bulkWrite').mockImplementation(async (operations, options) => { + if (!hasConcurrentChange) { + hasConcurrentChange = true; + await MongoApp.collection.updateOne( + { _id: appId }, + { $set: { publishedVersionId: concurrentVersionId } } + ); + } + return originalAppBulkWrite(operations, options); + }); + + const result = await runInitAppResourcesMigration({ + dryRun: false, + batchSize: 1, + writeBatchSize: 1 + }); + + expect(result).toMatchObject({ + stats: { + appsScanned: 1, + versionsScanned: 1, + appsUpdated: 0, + versionsUpdated: 1, + appsSkipped: 1, + versionsSkipped: 0 + } + }); + expect(await MongoApp.collection.findOne({ _id: appId })).toMatchObject({ + publishedVersionId: concurrentVersionId, + modules: [], + resourceRefs: { skillIds: ['legacy-skill'] } + }); + }); + + it('creates a published version only when the app has no versions', async () => { + await insertLegacyAppWithoutVersions([ + { + id: 'react-flow-node-id', + nodeId: 'node-1', + flowNodeType: 'workflowStart', + name: 'Start', + inputs: [], + outputs: [] + } + ]); + + const result = await runInitAppResourcesMigration({ dryRun: false }); + + expect(result).toMatchObject({ + stats: { + appsScanned: 1, + versionsScanned: 0, + appsUpdated: 1, + versionsUpdated: 1 + } + }); + const createdVersion = await MongoAppVersion.collection.findOne({ appId }); + expect(createdVersion).toMatchObject({ + isPublish: true, + nodes: [ + { + nodeId: 'node-1', + flowNodeType: 'workflowStart', + name: 'Start', + inputs: [], + outputs: [] + } + ], + edges: [], + chatConfig: {}, + resources: [{ type: 'skill', id: 'legacy-skill' }] + }); + expect(createdVersion?.nodes?.[0]).not.toHaveProperty('id'); + expect(await MongoApp.collection.findOne({ _id: appId })).toMatchObject({ + publishedVersionId: createdVersion?._id + }); + expect(await MongoApp.countDocuments({ resourceRefs: { $exists: true } })).toBe(0); + }); + + it('removes a generated version when a zero-version app changes before pointer backfill', async () => { + await insertLegacyAppWithoutVersions(); + + const originalAppBulkWrite = MongoApp.collection.bulkWrite.bind(MongoApp.collection); + let hasConcurrentChange = false; + vi.spyOn(MongoApp.collection, 'bulkWrite').mockImplementation(async (operations, options) => { + if (!hasConcurrentChange) { + hasConcurrentChange = true; + await MongoApp.collection.updateOne( + { _id: appId }, + { $set: { modules: [{ nodeId: 'changed-after-read' }] } } + ); + } + return originalAppBulkWrite(operations, options); + }); + + const result = await runInitAppResourcesMigration({ + dryRun: false, + batchSize: 1, + writeBatchSize: 1 + }); + + expect(result).toMatchObject({ + stats: { + appsUpdated: 0, + versionsUpdated: 0, + appsSkipped: 1 + } + }); + expect(await MongoAppVersion.countDocuments({ appId })).toBe(0); + expect(await MongoApp.collection.findOne({ _id: appId })).toMatchObject({ + modules: [{ nodeId: 'changed-after-read' }], + resourceRefs: { skillIds: ['legacy-skill'] } + }); + }); +}); diff --git a/projects/app/test/pages/api/support/mcp/server/toolList.test.ts b/projects/app/test/pages/api/support/mcp/server/toolList.test.ts index 6b8d03d5d2e7..95ec1e97bf2b 100644 --- a/projects/app/test/pages/api/support/mcp/server/toolList.test.ts +++ b/projects/app/test/pages/api/support/mcp/server/toolList.test.ts @@ -312,7 +312,8 @@ describe('callMcpServerTool', () => { } ], edges: [], - chatConfig: {} + chatConfig: {}, + resources: [] } as any); vi.mocked(getRunningUserInfoByTmbId).mockResolvedValue({ username: 'user', diff --git a/projects/app/test/service/core/app/controller.test.ts b/projects/app/test/service/core/app/controller.test.ts index e337e90ac10a..7af36c198a9a 100644 --- a/projects/app/test/service/core/app/controller.test.ts +++ b/projects/app/test/service/core/app/controller.test.ts @@ -1,8 +1,11 @@ import { describe, expect, it, vi } from 'vitest'; +import { beforeUpdateAppFormat } from '@fastgpt/service/core/app/controller'; +import { extractAppResources } from '@fastgpt/service/core/app/resources'; import { - beforeUpdateAppFormat, - validatePublishAppAgentSkillReadPermissions -} from '@fastgpt/service/core/app/controller'; + checkAppResourceReadPermissions, + resolveAppResourcesByPermission +} from '@fastgpt/service/support/permission/app/resource'; +import { AppTypeEnum } from '@fastgpt/global/core/app/constants'; import { FlowNodeInputTypeEnum, FlowNodeTypeEnum @@ -11,10 +14,21 @@ import { SystemToolSecretInputTypeEnum } from '@fastgpt/global/core/app/tool/sys import { NodeInputKeyEnum } from '@fastgpt/global/core/workflow/constants'; import type { StoreNodeItemType } from '@fastgpt/global/core/workflow/type/node'; import { MongoAgentSkills } from '@fastgpt/service/core/ai/skill/model/schema'; +import { MongoApp } from '@fastgpt/service/core/app/schema'; +import { MongoAppVersion } from '@fastgpt/service/core/app/version/schema'; import { AgentSkillSourceEnum } from '@fastgpt/global/core/ai/skill/constants'; import { getNanoid } from '@fastgpt/global/common/string/tools'; import { getUser } from '@test/datas/users'; +import { AppErrEnum } from '@fastgpt/global/common/error/code/app'; import { SkillErrEnum } from '@fastgpt/global/common/error/code/skill'; +import { MongoDataset } from '@fastgpt/service/core/dataset/schema'; +import { DatasetTypeEnum } from '@fastgpt/global/core/dataset/constants'; +import { MongoResourcePermission } from '@fastgpt/service/support/permission/schema'; +import { + PerResourceTypeEnum, + ReadPermissionVal +} from '@fastgpt/global/support/permission/constant'; +import { Types } from '@fastgpt/service/common/mongo'; const mocks = vi.hoisted(() => ({ getClientToolPreviewNode: vi.fn() @@ -438,7 +452,7 @@ describe('beforeUpdateAppFormat', () => { }); }); -describe('validatePublishAppAgentSkillReadPermissions', () => { +describe('checkAppResourceReadPermissions', () => { it('发布应用时校验静态绑定的 Agent Skill 读权限', async () => { const owner = await getUser(`publish-skill-owner-${getNanoid(6)}`); const member = await getUser(`publish-skill-member-${getNanoid(6)}`, owner.teamId); @@ -465,17 +479,179 @@ describe('validatePublishAppAgentSkillReadPermissions', () => { ]; await expect( - validatePublishAppAgentSkillReadPermissions({ - nodes, + checkAppResourceReadPermissions({ + resources: extractAppResources({ nodes }), tmbId: member.tmbId }) ).rejects.toBe(SkillErrEnum.unAuthSkill); await expect( - validatePublishAppAgentSkillReadPermissions({ - nodes, + checkAppResourceReadPermissions({ + resources: extractAppResources({ nodes }), + tmbId: owner.tmbId + }) + ).resolves.toBeUndefined(); + }); + + it('缺失 inheritPermission 的历史 App 和 Dataset 继续继承父级读权限', async () => { + const owner = await getUser(`legacy-inherit-owner-${getNanoid(6)}`); + const member = await getUser(`legacy-inherit-member-${getNanoid(6)}`, owner.teamId); + const [appFolder, datasetFolder] = await Promise.all([ + MongoApp.create({ + name: 'App folder', + type: AppTypeEnum.folder, + teamId: owner.teamId, tmbId: owner.tmbId + }), + MongoDataset.create({ + name: 'Dataset folder', + type: DatasetTypeEnum.folder, + teamId: owner.teamId, + tmbId: owner.tmbId + }) + ]); + const childAppId = new Types.ObjectId(); + const childDatasetId = new Types.ObjectId(); + + // 绕过 Schema 默认值,模拟升级前未写 inheritPermission 的历史数据。 + await Promise.all([ + MongoApp.collection.insertOne({ + _id: childAppId, + name: 'Legacy child app', + type: AppTypeEnum.workflow, + parentId: appFolder._id, + teamId: new Types.ObjectId(owner.teamId), + tmbId: new Types.ObjectId(owner.tmbId) + }), + MongoDataset.collection.insertOne({ + _id: childDatasetId, + name: 'Legacy child dataset', + type: DatasetTypeEnum.dataset, + parentId: datasetFolder._id, + teamId: new Types.ObjectId(owner.teamId), + tmbId: new Types.ObjectId(owner.tmbId) + }), + MongoResourcePermission.create([ + { + resourceType: PerResourceTypeEnum.app, + resourceId: appFolder._id, + teamId: owner.teamId, + tmbId: member.tmbId, + permission: ReadPermissionVal + }, + { + resourceType: PerResourceTypeEnum.dataset, + resourceId: datasetFolder._id, + teamId: owner.teamId, + tmbId: member.tmbId, + permission: ReadPermissionVal + } + ]) + ]); + + await expect( + checkAppResourceReadPermissions({ + resources: [ + { type: 'agent', id: String(childAppId) }, + { type: 'dataset', id: String(childDatasetId) } + ], + tmbId: member.tmbId }) ).resolves.toBeUndefined(); }); }); + +describe('resolveAppResourcesByPermission', () => { + it('does not recheck resources already present in the draft baseline', async () => { + const owner = await getUser(`draft-resource-owner-${getNanoid(6)}`); + const member = await getUser(`draft-resource-member-${getNanoid(6)}`, owner.teamId); + const workflowApp = await MongoApp.create({ + name: 'Workflow app', + type: AppTypeEnum.workflow, + modules: [], + edges: [], + teamId: owner.teamId, + tmbId: owner.tmbId + }); + const toolset = await MongoApp.create({ + name: 'Protected toolset', + type: AppTypeEnum.mcpToolSet, + modules: [], + edges: [], + teamId: owner.teamId, + tmbId: owner.tmbId + }); + const baselineResource = { type: 'tool' as const, id: String(toolset._id) }; + await MongoAppVersion.create({ + appId: workflowApp._id, + tmbId: owner.tmbId, + nodes: [], + edges: [], + resources: [baselineResource] + }); + + await expect( + resolveAppResourcesByPermission({ + appId: String(workflowApp._id), + extracted: [baselineResource], + tmbId: member.tmbId, + blockOnUnauthorized: true + }) + ).resolves.toEqual([baselineResource]); + + await expect( + resolveAppResourcesByPermission({ + appId: String(workflowApp._id), + extracted: [{ type: 'tool', id: String(workflowApp._id) }], + tmbId: member.tmbId, + blockOnUnauthorized: false + }) + ).resolves.toEqual([]); + + await expect( + resolveAppResourcesByPermission({ + appId: String(workflowApp._id), + extracted: [baselineResource, { type: 'tool', id: String(workflowApp._id) }], + tmbId: member.tmbId, + blockOnUnauthorized: true + }) + ).rejects.toBe(AppErrEnum.unAuthApp); + }); + + it('treats an invalid stored snapshot as empty for permission checks', async () => { + const owner = await getUser(`invalid-draft-resource-owner-${getNanoid(6)}`); + const member = await getUser(`invalid-draft-resource-member-${getNanoid(6)}`, owner.teamId); + const workflowApp = await MongoApp.create({ + name: 'Workflow app with invalid snapshot', + type: AppTypeEnum.workflow, + modules: [], + edges: [], + teamId: owner.teamId, + tmbId: owner.tmbId + }); + const protectedToolset = await MongoApp.create({ + name: 'Protected toolset', + type: AppTypeEnum.mcpToolSet, + modules: [], + edges: [], + teamId: owner.teamId, + tmbId: owner.tmbId + }); + await MongoAppVersion.create({ + appId: workflowApp._id, + tmbId: owner.tmbId, + nodes: [], + edges: [], + resources: [{ invalid: true } as any] + }); + + await expect( + resolveAppResourcesByPermission({ + appId: String(workflowApp._id), + extracted: [{ type: 'tool', id: String(protectedToolset._id) }], + tmbId: member.tmbId, + blockOnUnauthorized: true + }) + ).rejects.toBe(AppErrEnum.unAuthApp); + }); +}); diff --git a/projects/app/test/service/core/app/rewriteAppWorkflowToDetail.test.ts b/projects/app/test/service/core/app/rewriteAppWorkflowToDetail.test.ts index 33c4d2faa240..735973568052 100644 --- a/projects/app/test/service/core/app/rewriteAppWorkflowToDetail.test.ts +++ b/projects/app/test/service/core/app/rewriteAppWorkflowToDetail.test.ts @@ -17,6 +17,7 @@ import type { } from '@fastgpt/global/core/app/formEdit/type'; import { getNanoid } from '@fastgpt/global/common/string/tools'; import { getUser } from '@test/datas/users'; +import { AppErrEnum } from '@fastgpt/global/common/error/code/app'; const { getClientToolPreviewNodeMock, authAppByTmbIdMock } = vi.hoisted(() => ({ getClientToolPreviewNodeMock: vi.fn(), @@ -42,6 +43,11 @@ vi.mock('@fastgpt/service/support/permission/app/auth', async (importOriginal) = const { rewriteAppWorkflowToDetail } = await import('@fastgpt/service/core/app/utils'); describe('rewriteAppWorkflowToDetail - current workflow tool inputs', () => { + beforeEach(() => { + getClientToolPreviewNodeMock.mockReset(); + authAppByTmbIdMock.mockReset(); + }); + it('keeps the baseline invalid placeholder and error detail', async () => { getClientToolPreviewNodeMock.mockRejectedValue(new Error('Tool deleted')); const nodes = [ @@ -87,6 +93,46 @@ describe('rewriteAppWorkflowToDetail - current workflow tool inputs', () => { expect(tool.pluginData.error).toContain('Tool deleted'); }); + it('checks snapshot-external personal tools before loading preview metadata', async () => { + const toolAppId = '507f1f77bcf86cd799439011'; + authAppByTmbIdMock.mockRejectedValue(new Error('not allowed')); + const nodes = [ + { + nodeId: 'agent-1', + flowNodeType: FlowNodeTypeEnum.agent, + name: 'Agent', + inputs: [ + { + key: NodeInputKeyEnum.selectedTools, + value: [{ id: toolAppId, config: {} }] + } + ], + outputs: [] + } as StoreNodeItemType + ]; + + await rewriteAppWorkflowToDetail({ + nodes, + teamId: 'team-1', + ownerTmbId: 'owner-tmb', + viewerTmbId: 'viewer-tmb', + isRoot: false + }); + + const tool = (nodes[0].inputs[0].value as any)[0]; + expect(authAppByTmbIdMock).toHaveBeenCalledWith({ + tmbId: 'viewer-tmb', + appId: toolAppId, + per: expect.anything(), + isRoot: false + }); + expect(getClientToolPreviewNodeMock).not.toHaveBeenCalled(); + expect(tool.pluginData).toMatchObject({ + error: AppErrEnum.unAuthApp, + permissionDenied: true + }); + }); + it('普通节点不投影 customVariable 输入', async () => { const input = { key: 'externalVariable', @@ -611,7 +657,8 @@ describe('rewriteAppWorkflowToDetail - agent skills', () => { nodes, teamId: user.teamId, ownerTmbId: user.tmbId, - isRoot: false + isRoot: false, + resources: [{ type: 'skill', id: String(activeSkill._id) }] }); const rewrittenSkills = nodes[0].inputs.find((input) => input.key === NodeInputKeyEnum.skills) @@ -623,13 +670,16 @@ describe('rewriteAppWorkflowToDetail - agent skills', () => { name: 'Current Skill Name', description: 'Current skill description', avatar: 'current-avatar', - isDeleted: false + isDeleted: false, + permissionDenied: false }, { skillId: String(deletedSkill._id), name: 'Deleted Snapshot', description: 'Deleted snapshot description', - isDeleted: true + avatar: undefined, + isDeleted: true, + permissionDenied: false } ]); }); @@ -1274,7 +1324,8 @@ describe('rewriteAppWorkflowToDetail - agent skills', () => { nodes, teamId: user.teamId, ownerTmbId: user.tmbId, - isRoot: false + isRoot: false, + resources: [{ type: 'dataset', id: String(dataset._id) }] }); expect(datasetSelectInput.value).toEqual(referenceValue); @@ -1325,7 +1376,8 @@ describe('rewriteAppWorkflowToDetail - agent skills', () => { nodes, teamId: user.teamId, ownerTmbId: user.tmbId, - isRoot: false + isRoot: false, + resources: [{ type: 'dataset', id: String(dataset._id) }] }); const rewrittenDatasetParams = nodes[0].inputs.find( @@ -1341,7 +1393,8 @@ describe('rewriteAppWorkflowToDetail - agent skills', () => { modelId: resolvedEmbeddingModel.modelId, model: resolvedEmbeddingModel.model }), - isDeleted: false + isDeleted: false, + permissionDenied: false } ]); }); @@ -1376,7 +1429,8 @@ describe('rewriteAppWorkflowToDetail - agent skills', () => { nodes, teamId: user.teamId, ownerTmbId: user.tmbId, - isRoot: false + isRoot: false, + resources: [{ type: 'dataset', id: String(dataset._id) }] }); expect( @@ -1390,7 +1444,8 @@ describe('rewriteAppWorkflowToDetail - agent skills', () => { modelId: resolvedEmbeddingModel.modelId, model: resolvedEmbeddingModel.model }), - isDeleted: false + isDeleted: false, + permissionDenied: false } ]); }); diff --git a/projects/app/test/service/core/app/scheduleTrigger.test.ts b/projects/app/test/service/core/app/scheduleTrigger.test.ts index 0df51164c9ea..888a4aa64120 100644 --- a/projects/app/test/service/core/app/scheduleTrigger.test.ts +++ b/projects/app/test/service/core/app/scheduleTrigger.test.ts @@ -22,12 +22,16 @@ const mocks = vi.hoisted(() => ({ getNextTimeByCronStringAndTimezone: vi.fn() })); -vi.mock('@fastgpt/service/common/logger', () => ({ - getLogger: () => ({ - info: vi.fn(), - error: vi.fn() - }) -})); +vi.mock('@fastgpt/service/common/logger', () => { + const logCategory = new Proxy({}, { get: () => logCategory }); + return { + LogCategories: logCategory, + getLogger: () => ({ + info: vi.fn(), + error: vi.fn() + }) + }; +}); vi.mock('@fastgpt/service/support/wallet/sub/utils', () => ({ getTeamPlanStatus: vi.fn(async () => ({ standard: { maxUploadFileCount: 20 } })) @@ -105,7 +109,8 @@ describe('getScheduleTriggerApp', () => { versionId: 'version-id', nodes: [{ nodeId: 'start' }], edges: [], - chatConfig: { variables: [] } + chatConfig: { variables: [] }, + resources: [] }); mocks.getWorkflowEntryNodeIds.mockReturnValue(['start']); mocks.storeNodes2RuntimeNodes.mockReturnValue([{ nodeId: 'runtime-start' }]); diff --git a/projects/app/test/service/core/sandbox/access.test.ts b/projects/app/test/service/core/sandbox/access.test.ts index 03dc9910cee0..9375a40b160b 100644 --- a/projects/app/test/service/core/sandbox/access.test.ts +++ b/projects/app/test/service/core/sandbox/access.test.ts @@ -13,6 +13,7 @@ const mocks = vi.hoisted(() => ({ createAgentSandboxPermissionDeniedError: vi.fn(), findAppById: vi.fn(), getAppLatestVersion: vi.fn(), + getAppDraftWorkflow: vi.fn(), resolveAppSandboxAvailability: vi.fn() })); @@ -33,7 +34,8 @@ vi.mock('@fastgpt/service/core/app/schema', () => ({ })); vi.mock('@fastgpt/service/core/app/version/controller', () => ({ - getAppLatestVersion: mocks.getAppLatestVersion + getAppLatestVersion: mocks.getAppLatestVersion, + getAppDraftWorkflow: mocks.getAppDraftWorkflow })); import { @@ -67,6 +69,7 @@ describe('resolveSandboxSessionAvailability', () => { lean: vi.fn().mockResolvedValue({ _id: 'app_1', modules: [] }) }); mocks.getAppLatestVersion.mockResolvedValue({ nodes: [], edges: [], chatConfig: {} }); + mocks.getAppDraftWorkflow.mockResolvedValue({ nodes: [], edges: [], chatConfig: {} }); }); it('short-circuits before App queries when the system feature is disabled', async () => { @@ -109,8 +112,10 @@ describe('resolveSandboxSessionAvailability', () => { ); it('falls back to current draft nodes only for legacy Chat Test sessions', async () => { - mocks.findAppById.mockReturnValueOnce({ - lean: vi.fn().mockResolvedValue({ _id: 'app_1', modules: [sandboxNode(true)] }) + mocks.getAppDraftWorkflow.mockResolvedValueOnce({ + nodes: [sandboxNode(true)], + edges: [], + chatConfig: {} }); await resolveSandboxSessionAvailability({ @@ -122,6 +127,7 @@ describe('resolveSandboxSessionAvailability', () => { appEnabled: true, teamId: 'team_1' }); + expect(mocks.getAppDraftWorkflow).toHaveBeenCalled(); expect(mocks.getAppLatestVersion).not.toHaveBeenCalled(); }); diff --git a/projects/app/test/web/core/app/utils.test.ts b/projects/app/test/web/core/app/utils.test.ts index 0c32dba34fb9..5fb3319e4c1e 100644 --- a/projects/app/test/web/core/app/utils.test.ts +++ b/projects/app/test/web/core/app/utils.test.ts @@ -1,9 +1,5 @@ import { describe, expect, it } from 'vitest'; -import { - addModelNamesToAppForm, - filterSensitiveFormData, - getAppQGuideCustomURL -} from '@/web/core/app/utils'; +import { addModelNamesToAppForm, filterSensitiveFormData } from '@/web/core/app/utils'; import { appWorkflow2Form, form2AppWorkflow @@ -438,32 +434,6 @@ describe('addModelNamesToAppForm', () => { }); }); -describe('getAppQGuideCustomURL', () => { - it('should get custom URL from app detail', () => { - const appDetail = { - modules: [], - chatConfig: { - chatInputGuide: { - open: true, - customUrl: 'https://example.com' - } - } - } as any; - - const result = getAppQGuideCustomURL(appDetail); - expect(result).toBe('https://example.com'); - }); - - it('should return empty string if no custom URL found', () => { - const appDetail = { - modules: [] - } as any; - - const result = getAppQGuideCustomURL(appDetail); - expect(result).toBe(''); - }); -}); - describe('appWorkflow2AgentForm', () => { const mockT = (str: string) => str; diff --git a/projects/app/test/web/core/app/workflow/utils.test.ts b/projects/app/test/web/core/app/workflow/utils.test.ts index 12a6113b96e2..17257a469646 100644 --- a/projects/app/test/web/core/app/workflow/utils.test.ts +++ b/projects/app/test/web/core/app/workflow/utils.test.ts @@ -626,6 +626,106 @@ describe('checkWorkflowNodeIssues', () => { expect(result.toolCall.map((issue) => issue.code)).toContain('tool_call_empty'); }); + it('reports deleted dataset or skill references via resource_missing', () => { + const datasetNode = makeNode('dataset', FlowNodeTypeEnum.answerNode, { + inputs: [ + { + key: NodeInputKeyEnum.datasetSelectList, + label: '知识库', + renderTypeList: [FlowNodeInputTypeEnum.custom], + value: [ + { datasetId: 'd1', name: 'ok', isDeleted: false }, + { datasetId: 'd2', name: 'gone', isDeleted: true } + ] + } + ] + }); + const skillNode = makeNode('skill', FlowNodeTypeEnum.answerNode, { + inputs: [ + { + key: NodeInputKeyEnum.skills, + label: '技能', + renderTypeList: [FlowNodeInputTypeEnum.custom], + value: [{ skillId: 's2', name: 'gone', isDeleted: true }] + } + ] + }); + + const result = checkWorkflowNodeIssues({ + nodes: [startNode, datasetNode, skillNode], + edges: [ + { id: 'e1', source: 'start', target: 'dataset', type: EDGE_TYPE }, + { id: 'e2', source: 'start', target: 'skill', type: EDGE_TYPE } + ] + }); + + expect(result.dataset.map((issue) => issue.code)).toContain('resource_missing'); + expect(result.skill.map((issue) => issue.code)).toContain('resource_missing'); + }); + + it('focuses resources with an explicit ACL denial marker', () => { + const datasetNode = makeNode('dataset', FlowNodeTypeEnum.answerNode, { + inputs: [ + { + key: NodeInputKeyEnum.datasetSelectList, + label: '知识库', + renderTypeList: [FlowNodeInputTypeEnum.custom], + value: [{ datasetId: 'd1', name: 'no-perm', permissionDenied: true }] + } + ] + }); + const toolNode = makeNode('tool', FlowNodeTypeEnum.appModule, { + pluginData: { permissionDenied: true } as any + }); + + const result = checkWorkflowNodeIssues({ + nodes: [startNode, datasetNode, toolNode], + edges: [ + { id: 'e1', source: 'start', target: 'dataset', type: EDGE_TYPE }, + { id: 'e2', source: 'start', target: 'tool', type: EDGE_TYPE } + ] + }); + + expect((result.dataset ?? []).map((issue) => issue.code)).toContain('resource_no_permission'); + expect((result.tool ?? []).map((issue) => issue.code)).toContain('resource_no_permission'); + }); + + it('reports unavailable datasets nested in Agent datasetParams', () => { + const agentNode = makeNode('agent', FlowNodeTypeEnum.agent, { + inputs: [ + { + key: NodeInputKeyEnum.datasetParams, + label: '知识库配置', + renderTypeList: [FlowNodeInputTypeEnum.custom], + value: { + datasets: [ + { datasetId: 'd1', name: 'no-perm', permissionDenied: true }, + { datasetId: 'd2', name: 'gone', isDeleted: true } + ] + } + } + ] + }); + + const result = checkWorkflowNodeIssues({ + nodes: [startNode, agentNode], + edges: [{ id: 'e1', source: 'start', target: 'agent', type: EDGE_TYPE }] + }); + + expect(result.agent).toEqual( + expect.arrayContaining([ + expect.objectContaining({ + code: 'resource_no_permission', + inputKey: NodeInputKeyEnum.datasetParams + }), + expect.objectContaining({ + code: 'resource_missing', + inputKey: NodeInputKeyEnum.datasetParams + }) + ]) + ); + }); + /** * 使用 packages/global 真实节点模板构造 inputs,避免手写 valueType 掩盖模板默认值。 */