From 6803ea55db15e273016ded6f3d270b658457d47f Mon Sep 17 00:00:00 2001 From: thankyou <> Date: Tue, 13 Oct 2026 10:50:25 -0500 Subject: [PATCH 1/2] fix(runtime): let a subscription ClaudeAgent use CLAUDE_CODE_OAUTH_TOKEN on headless hosts (#1266) Claude Code's documented non-interactive login is a long-lived token from `claude setup-token`, passed as CLAUDE_CODE_OAUTH_TOKEN. Ultrafuzz never gave it to the controller, and when allowlisted it was blanked for every agent because no route owned it, so on a headless host every subscription-mode Claude task failed the engine preflight with "Claude Code is not logged in". A subscription ClaudeAgent now consumes the token the way an api-key agent consumes its key: the controller receives it and names it as the Claude route's provider credential, which keeps it through resume and blanks it for every other model child. The built-in provider credential list blanks the token and its file-descriptor variant by default, and the Claude adapter restores it only in subscription mode. The adapter carries it into the command environment, which Smithers applies last when it spawns. `ultrafuzz validate` now warns with AGENT_ENV_ALLOWLIST_CREDENTIAL_BLANKED when ULTRAFUZZ_AGENT_ENV_ALLOWLIST names a credential that no selected agent uses, since every agent would receive it blank. Co-Authored-By: Claude Opus 5.5 --- CHANGELOG.md | 1 + docs/config.md | 8 ++ docs/security.md | 5 +- .../src/provider-credential-environment.ts | 24 ++++ packages/runtime/src/start-run.ts | 17 +-- .../src/templates/smithers/agents/claude.tsx | 20 +++- .../templates/smithers/agents/environment.tsx | 4 + packages/runtime/src/validate.ts | 46 +++++++- .../test/provider-home-preflight.test.ts | 30 +++++ packages/runtime/test/runtime.test.ts | 106 +++++++++++++++++- 10 files changed, 248 insertions(+), 13 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 7705c004a..2f587e7d6 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -31,6 +31,7 @@ ### Other changes +- **[runtime] [docs]** A subscription-mode `ClaudeAgent` now runs on a headless host: `CLAUDE_CODE_OAUTH_TOKEN`, the long-lived login from `claude setup-token`, reaches Claude agents the way an API key reaches its own adapter, and every other model child has it blanked (#1266). Before, Ultrafuzz stripped it from every agent, even when allowlisted, and each Claude task failed the engine's preflight with "Claude Code is not logged in". `ultrafuzz validate` now warns with `AGENT_ENV_ALLOWLIST_CREDENTIAL_BLANKED` when `ULTRAFUZZ_AGENT_ENV_ALLOWLIST` names a credential that no selected agent uses, since every agent would receive it blank. - **[runtime]** Production copies of `proxy-addr`, `vue`, `source-map-js` and `prosemirror-view` move past the Critical advisory GHSA-jqcg-44mw-7w3h (`proxy-addr` trusted every client as a proxy when a trust subnet was an IPv4-mapped IPv6 address with a short prefix) and the High advisories GHSA-g2v6-rqmx-r4w6 (XSS in `@vue/server-renderer`), GHSA-68fv-2mgg-jv7q (event-loop denial of service in `source-map-js`) and GHSA-c8x8-7fp4-3x9w (XSS in `prosemirror-view` paste handling). All four reach Ultrafuzz only through the workflow engine: Express under `@modelcontextprotocol/sdk`, and the Milkdown editor in `@smthrs/ui`. In `pnpm-lock.yaml`, `proxy-addr` moves from 2.0.7 to 2.0.8, `vue` and its `@vue/*` packages from 3.5.40 to 3.5.42, `source-map-js` from 1.2.1 to 1.2.2 and `prosemirror-view` from 1.42.2 to 1.42.3, all within their parents' ranges. Like any install that changes the engine's dependency tree, `pnpm install` of this change moves the engine to a new directory: a run still in progress in that checkout can fail until `ultrafuzz resume` continues it, and long-lived processes such as the dashboard need a restart. The engine that `run` installs from npm does not read that lockfile; it resolves every open range as of a fixed instant, which moves from `2026-08-18T06:00:00Z` to `2026-10-01T00:00:00Z`. Newly launched runs therefore also get the fixed releases (`vue` 3.5.43 and `prosemirror-view` 1.42.6 there), and `fast-uri` 3.1.8 and `undici` 7.30.0, which clear seven High advisories in that install that earlier lockfile fixes did not reach (GHSA-5jgf-p345-68v8, GHSA-f65p-4m7j-42xc, GHSA-fph4-wmhf-6fwf, GHSA-jqff-g426-hqxp and GHSA-qw65-cvwx-89v3 in `fast-uri`, GHSA-rfgv-xxqx-mfg5 and GHSA-w293-vg96-wgc3 in `undici`). The new instant also moves 94 other packages of the roughly 700 in that install to their newest in-range release as of that date, including `react` and `react-dom` from 19.2.8 to 19.3.0 and `ai` from 7.0.66 to 7.0.126. - **[runtime]** The workspace copy of `@modelcontextprotocol/sdk` moves from 1.29.0 to 1.31.0, past the High advisory GHSA-6qxp-vccf-f47h (the SDK's OAuth client could send credentials to an authorization server chosen by the MCP server). The workflow engine that `run` installs already resolves 1.31.0 at its 2026-10-01 dependency cutoff. - **[runtime] [docs]** `validate`, and so `run`, now refuse an agent provider home that the agent adapters would refuse, before planning, with `PROVIDER_HOME_UNSAFE` naming the directory and the command that fixes it, such as `chmod 700 ~/.codex` (#1265). `ultrafuzz doctor` reports the same diagnostic. Claude Code and Codex create `~/.claude` and `~/.codex` from the umask, usually `0755` or `0775`, and the adapters accept only `0700` for a provider home and for the Ultrafuzz provider-home root above one. So any Codex user (the `init` default) on macOS or Linux, and any Claude user on Linux, whose CLI had created its home failed only once the engine rendered the workflow, after about 2 minutes of planning and execution-snapshot preparation, with an engine `INTERNAL_ERROR` (`provider homes must be operator-owned with private 0700 permissions`) that named no directory. The check covers every agent with a provider home, `OpenRouterAgent` included, and resolves it as the adapters do, from the launch environment's `HOME` or, when that environment has no `HOME`, from the operator's home directory. It also reports a relative `ULTRAFUZZ_PROVIDER_HOME_ROOT`, `CODEX_HOME`, `CLAUDE_CONFIG_DIR`, `KIMI_CODE_HOME` or `KIMI_SHARE_DIR`, which the adapters refuse, and a directory it cannot inspect; a directory you cannot search is reported with `chmod u+x` on that directory, and the printed command quotes a path that needs it. Ultrafuzz never changes these directories itself. diff --git a/docs/config.md b/docs/config.md index 7a71238a7..32d371b2b 100644 --- a/docs/config.md +++ b/docs/config.md @@ -140,6 +140,14 @@ for running multiple Claude subscriptions side by side. Set `auth = "api-key"` to bill against the Anthropic API using canonical `ANTHROPIC_API_KEY`. Both modes drive the `claude` CLI, so it must be installed either way; `auth` only changes how that CLI authenticates. +On a headless host such as a container or CI runner, where `claude auth login` +cannot open a browser, generate a long-lived token with `claude setup-token` +on any machine and export it as `CLAUDE_CODE_OAUTH_TOKEN` before `ultrafuzz +run`. In subscription mode `ClaudeAgent` receives it, with no +`ULTRAFUZZ_AGENT_ENV_ALLOWLIST` entry, and every other agent has it blanked. +Export it again before `ultrafuzz resume`. `ClaudeAgent` in `api-key` mode does +not receive it. + When a Claude profile sets `reasoning`, the generated adapter passes it to the current Claude Code CLI as `--effort `. Pin the model through the profile's `model` field rather than under `[agents.ClaudeAgent]`. diff --git a/docs/security.md b/docs/security.md index 64efdc7d7..d8c308734 100644 --- a/docs/security.md +++ b/docs/security.md @@ -253,6 +253,9 @@ children and omitted from unrelated Modal task secrets. A recognized provider-route prefix such as `AWS_` scopes the value to that route. Arbitrary unrecognized credentials are not supported and reach no model task. Stock agent API keys use their validated canonical `api_key_env`; the active adapter -restores only its own key. Use only a credential-free RPC endpoint when an RPC +restores only its own key. A subscription `ClaudeAgent` likewise restores only +`CLAUDE_CODE_OAUTH_TOKEN`, Claude Code's headless login. `ultrafuzz validate` +warns when an allowlisted credential is owned by no selected agent, since every +agent would receive it blank. Use only a credential-free RPC endpoint when an RPC URL must remain a global input. Do not add unrelated credentials merely to make them available to prompts. diff --git a/packages/runtime/src/provider-credential-environment.ts b/packages/runtime/src/provider-credential-environment.ts index 49c32b28f..c181b44b3 100644 --- a/packages/runtime/src/provider-credential-environment.ts +++ b/packages/runtime/src/provider-credential-environment.ts @@ -6,6 +6,30 @@ const ROUTE_ENV_PREFIXES: Readonly> = { KimiAgent: ["KIMI_", "MOONSHOT_"] }; +/** + * Claude Code's documented non-interactive login: a long-lived token from + * `claude setup-token`. A subscription-mode ClaudeAgent receives it the way an + * api-key agent receives its key, and every other model child has it blanked + * (#1266). + */ +export const CLAUDE_SUBSCRIPTION_TOKEN_ENVIRONMENT_VARIABLE = "CLAUDE_CODE_OAUTH_TOKEN"; + +/** The credential variables an agent's adapter reads for its configured authentication. */ +export function agentAuthenticationEnvironmentVariableNames( + agentRef: string, + agent: { auth?: string; apiKeyEnv?: string } +): string[] { + if (agent.auth === "api-key" && agent.apiKeyEnv !== undefined) { + return agentRef === "KimiAgent" && agent.apiKeyEnv === "KIMI_API_KEY" + ? [agent.apiKeyEnv, "MOONSHOT_API_KEY"] + : [agent.apiKeyEnv]; + } + if (agentRef === "ClaudeAgent" && agent.auth === "subscription") { + return [CLAUDE_SUBSCRIPTION_TOKEN_ENVIRONMENT_VARIABLE]; + } + return []; +} + export function isCredentialLikeEnvironmentVariableName(name: string): boolean { return isSensitiveEnvironmentName(name); } diff --git a/packages/runtime/src/start-run.ts b/packages/runtime/src/start-run.ts index 7b916080e..3b2fc4e83 100644 --- a/packages/runtime/src/start-run.ts +++ b/packages/runtime/src/start-run.ts @@ -52,6 +52,7 @@ import { type WorkflowLifecycleValue } from "./types.js"; import { planRun } from "./plan-run.js"; +import { agentAuthenticationEnvironmentVariableNames } from "./provider-credential-environment.js"; import { refreshRunPrompts } from "./prompt-refresh.js"; import { probeCommandsForExecution } from "./required-commands.js"; import { forgeGuardMetadata, prepareForgeGuardEnvironment } from "./forge-guard.js"; @@ -2047,10 +2048,11 @@ function agentCredentialEnvironmentVariableNames(config: ResolvedConfig, agentRe const activeAgentRefs = new Set(agentRefs); const names: string[] = []; for (const [agentRef, agent] of Object.entries(config.agents)) { - if (!activeAgentRefs.has(agentRef) || agent.auth !== "api-key" || agent.apiKeyEnv === undefined) continue; - assertCredentialEnvironmentVariableName(agent.apiKeyEnv); - names.push(agent.apiKeyEnv); - if (agentRef === "KimiAgent" && agent.apiKeyEnv === "KIMI_API_KEY") names.push("MOONSHOT_API_KEY"); + if (!activeAgentRefs.has(agentRef)) continue; + for (const name of agentAuthenticationEnvironmentVariableNames(agentRef, agent)) { + assertCredentialEnvironmentVariableName(name); + names.push(name); + } } return [...new Set(names)].sort(); } @@ -2065,10 +2067,9 @@ function agentEnvironmentVariableNames( if (activeAgentRefs.size > 0) names.push("ULTRAFUZZ_PROVIDER_HOME_ROOT", ...MODEL_ROUTE_PROXY_ENVIRONMENT_VARIABLES); for (const [agentRef, agent] of Object.entries(config.agents)) { if (!activeAgentRefs.has(agentRef)) continue; - if (agent.auth === "api-key" && agent.apiKeyEnv !== undefined) { - assertCredentialEnvironmentVariableName(agent.apiKeyEnv); - names.push(agent.apiKeyEnv); - if (agentRef === "KimiAgent" && agent.apiKeyEnv === "KIMI_API_KEY") names.push("MOONSHOT_API_KEY"); + for (const name of agentAuthenticationEnvironmentVariableNames(agentRef, agent)) { + assertCredentialEnvironmentVariableName(name); + names.push(name); } if (agentRef === "KimiAgent") { names.push("KIMI_BASE_URL", "KIMI_CODE_HOME", "KIMI_SHARE_DIR"); diff --git a/packages/runtime/src/templates/smithers/agents/claude.tsx b/packages/runtime/src/templates/smithers/agents/claude.tsx index 4eca4fb21..cefb657e9 100644 --- a/packages/runtime/src/templates/smithers/agents/claude.tsx +++ b/packages/runtime/src/templates/smithers/agents/claude.tsx @@ -57,9 +57,17 @@ export class CompatibleClaudeCodeAgent extends StatedFailureClaudeCodeAgent { this.opts.settingSources = "user"; const command = await super.buildCommand(params); try { + // The command environment is applied last when Smithers spawns, so the + // subscription login set on this agent (#1266) has to be carried into it, + // or the blanked provider credential would override it. + const login = this.opts.env?.CLAUDE_CODE_OAUTH_TOKEN; return { ...command, - env: workflowControlChildEnvironment(command.env, process.env, { agent: "ClaudeAgent" }) + env: workflowControlChildEnvironment( + login === undefined || login === "" ? command.env : { CLAUDE_CODE_OAUTH_TOKEN: login, ...command.env }, + process.env, + { agent: "ClaudeAgent" } + ) }; } catch (error) { await command.cleanup?.(); @@ -100,7 +108,15 @@ function claudeAuthOptions(): ClaudeAuthOptions { // ClaudeCodeAgent clears ANTHROPIC_API_KEY so the logged-in subscription (`claude -p`) // is used. configDir is always forwarded; without config_dir or ULTRAFUZZ_PROVIDER_HOME_ROOT, // resolveProviderHome falls back to $CLAUDE_CONFIG_DIR, then ~/.claude. - return { configDir }; + // On a headless host the subscription is a `claude setup-token` token instead of a login + // in the provider home. Every other model child has it blanked (#1266). + const token = process.env.CLAUDE_CODE_OAUTH_TOKEN; + return token === undefined || token === "" + ? { configDir } + : { + configDir, + env: { CLAUDE_CODE_OAUTH_TOKEN: workflowControlCredentialValue(token, "CLAUDE_CODE_OAUTH_TOKEN") } + }; } throw new Error(`unsupported ClaudeAgent auth mode in ultrafuzz.toml: ${auth}`); } diff --git a/packages/runtime/src/templates/smithers/agents/environment.tsx b/packages/runtime/src/templates/smithers/agents/environment.tsx index 3a9c8fc89..931774628 100644 --- a/packages/runtime/src/templates/smithers/agents/environment.tsx +++ b/packages/runtime/src/templates/smithers/agents/environment.tsx @@ -41,6 +41,10 @@ const BUILT_IN_PROVIDER_CREDENTIAL_ENVIRONMENT_VARIABLES = [ "ANTHROPIC_API_KEY", "ANTHROPIC_AUTH_TOKEN", "AZURE_OPENAI_API_KEY", + // Claude Code's `claude setup-token` login; only the subscription Claude + // adapter puts it back (#1266). + "CLAUDE_CODE_OAUTH_TOKEN", + "CLAUDE_CODE_OAUTH_TOKEN_FILE_DESCRIPTOR", "CODEX_API_KEY", "DEEPSEEK_API_KEY", "KIMI_API_KEY", diff --git a/packages/runtime/src/validate.ts b/packages/runtime/src/validate.ts index 904e41b9e..3b5bf06e8 100644 --- a/packages/runtime/src/validate.ts +++ b/packages/runtime/src/validate.ts @@ -26,6 +26,12 @@ import { effectiveAuditPolicy } from "./audit-profile-policy.js"; import { timeoutShadowingDiagnostics } from "./timeout-shadowing.js"; import { agentRegistryRegisters, inspectAgentRegistry } from "./agent-registry.js"; import { providerHomeDiagnostics, providerHomeProblems } from "./provider-home-preflight.js"; +import { + agentAuthenticationEnvironmentVariableNames, + CLAUDE_SUBSCRIPTION_TOKEN_ENVIRONMENT_VARIABLE, + isCredentialLikeEnvironmentVariableName, + routeOwnsCredentialLikeEnvironmentVariable +} from "./provider-credential-environment.js"; import { promptTextsForCatalog, transformPromptCatalogForRun, transformTopologyForRun } from "./topology-transform.js"; import { configDiagnostics, @@ -372,6 +378,7 @@ function evaluatePolicies( // The adapters refuse an unsafe provider home only once the engine renders the workflow, after // planning and the execution snapshot (#1265). Refusing it here fails `run` before either. const providerHomes = providerHomeDiagnostics(providerHomeProblems(selectedAgentRefs, config, env)); + const blankedCredentials = blankedAllowlistedCredentialDiagnostics(selectedAgentRefs, config, env); return { posture: { paths: postureFromDiagnostics("paths", "product files are written through project-local path guards", []), @@ -380,7 +387,7 @@ function evaluatePolicies( providerHomes.length === 0 ? "agent references resolve before launch" : "agent references resolve, but an agent's provider home would be refused at launch", - [...agentRegistry, ...providerHomes] + [...agentRegistry, ...providerHomes, ...blankedCredentials] ), trust: postureFromDiagnostics( "trust", @@ -440,3 +447,40 @@ function applyAgentOverrides(config: ResolvedConfig, input: ValidateProjectInput reasoning: input.reasoning }); } + +/** + * An allowlisted variable that looks like a credential reaches only the agent + * route that owns it. When no selected agent owns it, every agent receives it + * blank, which surfaces only as that agent's own authentication failure. Say + * so before launch (#1266). + */ +function blankedAllowlistedCredentialDiagnostics( + selectedAgentRefs: readonly string[], + config: ResolvedConfig, + env: Record +): RuntimeDiagnostic[] { + const consumed = new Set( + selectedAgentRefs.flatMap((agentRef) => { + const agent = config.agents[agentRef]; + return agent === undefined ? [] : agentAuthenticationEnvironmentVariableNames(agentRef, agent); + }) + ); + const diagnostics: RuntimeDiagnostic[] = []; + for (const name of new Set((env.ULTRAFUZZ_AGENT_ENV_ALLOWLIST ?? "").split(",").map((entry) => entry.trim()))) { + const upper = name.toUpperCase(); + if (name === "" || !isCredentialLikeEnvironmentVariableName(upper) || consumed.has(upper)) continue; + if (selectedAgentRefs.some((agentRef) => routeOwnsCredentialLikeEnvironmentVariable(agentRef, upper))) continue; + diagnostics.push({ + code: "AGENT_ENV_ALLOWLIST_CREDENTIAL_BLANKED", + message: + `${name} is in ULTRAFUZZ_AGENT_ENV_ALLOWLIST but looks like a credential that no selected agent uses, ` + + "so every agent receives it blank" + + (upper === CLAUDE_SUBSCRIPTION_TOKEN_ENVIRONMENT_VARIABLE + ? `; ClaudeAgent with auth = "subscription" receives ${CLAUDE_SUBSCRIPTION_TOKEN_ENVIRONMENT_VARIABLE} without the allowlist` + : ""), + severity: "warning", + source: "agents" + }); + } + return diagnostics; +} diff --git a/packages/runtime/test/provider-home-preflight.test.ts b/packages/runtime/test/provider-home-preflight.test.ts index 2d9188d9d..7d4f3bdf4 100644 --- a/packages/runtime/test/provider-home-preflight.test.ts +++ b/packages/runtime/test/provider-home-preflight.test.ts @@ -202,6 +202,36 @@ test("validate refuses a project whose agent would get an unsafe provider home, ); }); +test("validate warns when an allowlisted credential reaches no selected agent (#1266)", async () => { + const project = temporaryRoot("ufz-blanked-credential-project-"); + assert.equal(initProject({ projectRoot: project, force: true }).ok, true); + const home = privateHome(); + const env = { + HOME: home, + ULTRAFUZZ_AGENT_ENV_ALLOWLIST: "CLAUDE_CODE_OAUTH_TOKEN,MY_SERVICE_TOKEN,OPENAI_ORG_TOKEN,FORK_BLOCK", + CLAUDE_CODE_OAUTH_TOKEN: "claude-setup-token-login", + MY_SERVICE_TOKEN: "unrouted-credential", + OPENAI_ORG_TOKEN: "codex-routed-credential", + FORK_BLOCK: "19000000" + }; + const blanked = (result: Awaited>): string[] => + result.diagnostics + .filter(({ code }) => code === "AGENT_ENV_ALLOWLIST_CREDENTIAL_BLANKED") + .map(({ message }) => message.split(" ")[0] ?? ""); + + // Codex owns OPENAI_*; a plain value is not a credential; the others reach nobody. + const codex = await validateProject({ projectRoot: project, env, agent: "CodexAgent" }); + assert.deepEqual(blanked(codex).sort(), ["CLAUDE_CODE_OAUTH_TOKEN", "MY_SERVICE_TOKEN"]); + const claudeHint = codex.diagnostics.find(({ message }) => message.startsWith("CLAUDE_CODE_OAUTH_TOKEN")); + assert.match(claudeHint?.message ?? "", /auth = "subscription" receives CLAUDE_CODE_OAUTH_TOKEN/u); + assert.equal(codex.value?.policy_posture.agents.status, "warn"); + assert.equal(codex.ok, true, JSON.stringify(codex.diagnostics)); + + // A subscription Claude agent consumes its own login; with no Codex agent, OPENAI_* reaches nobody. + const claude = await validateProject({ projectRoot: project, env, agent: "ClaudeAgent" }); + assert.deepEqual(blanked(claude).sort(), ["MY_SERVICE_TOKEN", "OPENAI_ORG_TOKEN"]); +}); + // The prediction duplicates the adapters' rules, so check it against the adapter code itself. test( "predictedProviderHome matches the generated adapters' resolveProviderHome", diff --git a/packages/runtime/test/runtime.test.ts b/packages/runtime/test/runtime.test.ts index ce2dc608d..cb7596d12 100644 --- a/packages/runtime/test/runtime.test.ts +++ b/packages/runtime/test/runtime.test.ts @@ -156,6 +156,7 @@ if (runningUnderBun) process.env.ULTRAFUZZ_RUNTIME_MODULE ??= new URL("../src/in const SMITHERS_TEST_ENVIRONMENT_ALLOWLIST = [ "SMITHERS_FAKE_ADMISSION_TIMEOUT_LOG", "SMITHERS_FAKE_CONTEXT_LOG", + "SMITHERS_FAKE_CLAUDE_ENV_LOG", "SMITHERS_FAKE_DEEPSEEK_ENV_LOG", "SMITHERS_FAKE_ENV_LOG", "SMITHERS_FAKE_EXECUTED_AS_LOG", @@ -1616,6 +1617,12 @@ function readOpenRouterRetryFixtureJournal(pathname: string): OpenRouterRetryFix } async function loadGeneratedDeepSeekAgent(project: string): Promise<{ + createClaudeAgent(options?: Record): { + buildCommand(params: { prompt: string; cwd: string; options: Record }): Promise<{ + env?: Record; + cleanup?: () => void | Promise; + }>; + }; createDeepSeekAgent(options?: Record): { buildCommand(params: { prompt: string; cwd: string; options: Record }): Promise<{ env?: Record; @@ -1726,6 +1733,12 @@ async function loadGeneratedDeepSeekAgent(project: string): Promise<{ }; }; const claudeModule = (await import(pathToFileURL(path.join(fixture, "claude.mjs")).href)) as { + createClaudeAgent(options?: Record): { + buildCommand(params: { prompt: string; cwd: string; options: Record }): Promise<{ + env?: Record; + cleanup?: () => void | Promise; + }>; + }; CompatibleClaudeCodeAgent: new (options: Record) => { buildCommand(params: { prompt: string; @@ -1735,6 +1748,7 @@ async function loadGeneratedDeepSeekAgent(project: string): Promise<{ }; }; return { + createClaudeAgent: claudeModule.createClaudeAgent, createDeepSeekAgent: deepSeekModule.createDeepSeekAgent, CompatibleClaudeCodeAgent: claudeModule.CompatibleClaudeCodeAgent, DeepSeekClaudeCodeAgent: deepSeekModule.DeepSeekClaudeCodeAgent @@ -1874,6 +1888,9 @@ function fakeSmithersEnv(project: string): Record { 'if [ -n "$SMITHERS_FAKE_KIMI_ENV_LOG" ]; then', ' printf \'%s|%s|%s|%s|%s|%s|%s|%s\\n\' "$KIMI_API_KEY" "$MOONSHOT_API_KEY" "$KIMI_BASE_URL" "$KIMI_CODE_HOME" "$KIMI_SHARE_DIR" "$ULTRAFUZZ_KIMI_SHARED_AUTH_HOME" "$ULTRAFUZZ_KIMI_SESSION_HOME" "$ULTRAFUZZ_MODAL_REMOTE_ROOT" > "$SMITHERS_FAKE_KIMI_ENV_LOG"', "fi", + 'if [ -n "$SMITHERS_FAKE_CLAUDE_ENV_LOG" ]; then', + ' printf \'%s|%s\\n\' "$CLAUDE_CODE_OAUTH_TOKEN" "$ULTRAFUZZ_PROVIDER_CREDENTIAL_ENV_NAMES" > "$SMITHERS_FAKE_CLAUDE_ENV_LOG"', + "fi", 'if [ -n "$SMITHERS_FAKE_DEEPSEEK_ENV_LOG" ]; then', ' printf \'%s|%s\\n\' "$DEEPSEEK_API_KEY" "$ANTHROPIC_API_KEY" > "$SMITHERS_FAKE_DEEPSEEK_ENV_LOG"', "fi", @@ -6730,6 +6747,64 @@ for (const adapter of ["Claude", "DeepSeek"] as const) { }); } +bunAdapterTest( + "generated subscription Claude adapter receives the setup-token login no other model child sees (#1266)", + { timeout: 30_000 }, + async () => { + const project = tempProject(); + const init = initProject({ projectRoot: project, force: true }); + assert.equal(init.ok, true, JSON.stringify(init.diagnostics)); + const { createClaudeAgent } = await loadGeneratedDeepSeekAgent(project); + const { workflowControlChildEnvironment } = (await import( + pathToFileURL(path.join(project, "deepseek-agent-executable-test", "environment.mjs")).href + )) as { + workflowControlChildEnvironment( + additions?: Record, + source?: Record, + route?: { agent: string } + ): Record; + }; + const token = "sk-ant-oat01-headless-setup-token-fixture"; + const saved = { + CLAUDE_CODE_OAUTH_TOKEN: process.env.CLAUDE_CODE_OAUTH_TOKEN, + ULTRAFUZZ_CONFIG_PATH: process.env.ULTRAFUZZ_CONFIG_PATH, + ULTRAFUZZ_PROVIDER_HOME_ROOT: process.env.ULTRAFUZZ_PROVIDER_HOME_ROOT + }; + process.env.CLAUDE_CODE_OAUTH_TOKEN = token; + process.env.ULTRAFUZZ_CONFIG_PATH = path.join(project, "ultrafuzz.toml"); + process.env.ULTRAFUZZ_PROVIDER_HOME_ROOT = path.join(project, ".ultrafuzz", "provider-homes"); + try { + const command = await createClaudeAgent({ model: "claude-opus-4-8" }).buildCommand({ + prompt: "Contract only", + cwd: project, + options: {} + }); + try { + assert.equal(command.env?.CLAUDE_CODE_OAUTH_TOKEN, token); + } finally { + await command.cleanup?.(); + } + // Every other route has the login blanked, even when it is allowlisted. + const source = { + ...process.env, + ULTRAFUZZ_AGENT_ENV_ALLOWLIST: "CLAUDE_CODE_OAUTH_TOKEN", + CLAUDE_CODE_OAUTH_TOKEN_FILE_DESCRIPTOR: "3" + }; + for (const agent of ["CodexAgent", "KimiAgent", "OpenRouterAgent"]) { + const child = workflowControlChildEnvironment({}, source, { agent }); + assert.equal(child.CLAUDE_CODE_OAUTH_TOKEN, "", `${agent} must not receive the Claude login`); + assert.equal(child.CLAUDE_CODE_OAUTH_TOKEN_FILE_DESCRIPTOR, "", agent); + } + assert.equal(workflowControlChildEnvironment({}, source).CLAUDE_CODE_OAUTH_TOKEN, ""); + } finally { + for (const [name, value] of Object.entries(saved)) { + if (value === undefined) Reflect.deleteProperty(process.env, name); + else process.env[name] = value; + } + } + } +); + bunAdapterTest( "generated DeepSeek adapter uses the official endpoint and isolates Claude routing", { timeout: 30_000 }, @@ -11759,7 +11834,8 @@ test("startRun --agent does not carry the previous agent's model onto the new ag env: { ...fakeSmithersEnv(project), CLAUDE_CONFIG_DIR: claudeHome, SMITHERS_FAKE_ENV_LOG: environmentLog } }); assert.equal(run.ok, true, JSON.stringify(run.diagnostics)); - assert.equal(fs.readFileSync(environmentLog, "utf8"), `|||${claudeHome}||\n`); + // A subscription Claude run names its setup-token login as its provider credential (#1266). + assert.equal(fs.readFileSync(environmentLog, "utf8"), `|||${claudeHome}||CLAUDE_CODE_OAUTH_TOKEN\n`); const smithersTasks = JSON.parse( fs.readFileSync(path.join(run.value!.run_root, "smithers", "tasks.json"), "utf8") @@ -14571,6 +14647,34 @@ test("startRun forwards only the configured DeepSeek API key for DeepSeek runs", assert.equal(fs.readFileSync(deepSeekEnvironmentLog, "utf8"), "deepseek-agent-key|\n"); }); +test("startRun forwards the Claude setup-token login only to subscription Claude runs (#1266)", async () => { + const project = tempProject(); + initProject({ projectRoot: project, force: true }); + writeSmallTopology(project); + const claudeEnvironmentLog = path.join(project, "smithers-claude-environment.log"); + const env = { + ...fakeSmithersEnv(project), + SMITHERS_FAKE_CLAUDE_ENV_LOG: claudeEnvironmentLog, + CLAUDE_CODE_OAUTH_TOKEN: "claude-setup-token-login" + }; + + // The controller carries the login and names it as the Claude route's own + // credential, which keeps it from every other model child and through resume. + const claude = await startRun({ + projectRoot: project, + runId: "claude-subscription-token", + agent: "ClaudeAgent", + env + }); + assert.equal(claude.ok, true, JSON.stringify(claude.diagnostics)); + assert.equal(fs.readFileSync(claudeEnvironmentLog, "utf8"), "claude-setup-token-login|CLAUDE_CODE_OAUTH_TOKEN\n"); + + // A run without a subscription Claude agent never receives it. + const codex = await startRun({ projectRoot: project, runId: "codex-ignores-claude-token", agent: "CodexAgent", env }); + assert.equal(codex.ok, true, JSON.stringify(codex.diagnostics)); + assert.equal(fs.readFileSync(claudeEnvironmentLog, "utf8"), "|OPENAI_API_KEY\n"); +}); + test("startRun preflights and isolates the dedicated OpenRouter credential while preserving the model ID", async () => { const project = tempProject(); initProject({ projectRoot: project, force: true }); From ab8866663add2b823f141cd85b519c87dde6a479 Mon Sep 17 00:00:00 2001 From: thankyou <> Date: Wed, 14 Oct 2026 11:23:17 -0500 Subject: [PATCH 2/2] fix(runtime): warn when an allowlisted built-in provider key reaches no agent (#1266) The validate warning skipped any allowlisted credential under a selected route's prefix, so subscription-mode Codex with OPENAI_API_KEY allowlisted gave no warning. A route never restores a built-in provider credential from the allowlist (workflowControlChildEnvironment blanks them, and subscription Codex blanks OPENAI_API_KEY itself); only the provider's own configured authentication does. Built-in credentials no longer count as route-owned for the warning. The built-in provider credential list now lives in provider-credential-environment.ts, and the agent environment template loads it from @ultrafuzz/runtime with the classification functions it already loads, instead of keeping its own copy. Co-Authored-By: Claude Opus 5.5 --- .../src/provider-credential-environment.ts | 21 ++++++++++++++ .../templates/smithers/agents/environment.tsx | 28 ++++++------------- packages/runtime/src/validate.ts | 8 +++++- .../test/provider-home-preflight.test.ts | 22 +++++++++++++++ 4 files changed, 59 insertions(+), 20 deletions(-) diff --git a/packages/runtime/src/provider-credential-environment.ts b/packages/runtime/src/provider-credential-environment.ts index c181b44b3..7b87c4873 100644 --- a/packages/runtime/src/provider-credential-environment.ts +++ b/packages/runtime/src/provider-credential-environment.ts @@ -6,6 +6,27 @@ const ROUTE_ENV_PREFIXES: Readonly> = { KimiAgent: ["KIMI_", "MOONSHOT_"] }; +/** + * Provider credentials every model child has blanked. Only the adapter that + * owns one restores it, from its configured authentication; an + * ULTRAFUZZ_AGENT_ENV_ALLOWLIST entry never does. + */ +export const BUILT_IN_PROVIDER_CREDENTIAL_ENVIRONMENT_VARIABLES = [ + "ANTHROPIC_API_KEY", + "ANTHROPIC_AUTH_TOKEN", + "AZURE_OPENAI_API_KEY", + // Claude Code's `claude setup-token` login; only the subscription Claude + // adapter puts it back (#1266). + "CLAUDE_CODE_OAUTH_TOKEN", + "CLAUDE_CODE_OAUTH_TOKEN_FILE_DESCRIPTOR", + "CODEX_API_KEY", + "DEEPSEEK_API_KEY", + "KIMI_API_KEY", + "MOONSHOT_API_KEY", + "OPENAI_API_KEY", + "OPENROUTER_API_KEY" +] as const; + /** * Claude Code's documented non-interactive login: a long-lived token from * `claude setup-token`. A subscription-mode ClaudeAgent receives it the way an diff --git a/packages/runtime/src/templates/smithers/agents/environment.tsx b/packages/runtime/src/templates/smithers/agents/environment.tsx index 931774628..5aad849b8 100644 --- a/packages/runtime/src/templates/smithers/agents/environment.tsx +++ b/packages/runtime/src/templates/smithers/agents/environment.tsx @@ -4,13 +4,18 @@ import { fileURLToPath } from "node:url"; export const PROVIDER_SCOPED_SENSITIVE_ENVIRONMENT_CAPABILITY = "ultrafuzz.provider-scoped-sensitive-environment.v1" as const; -// Credential classification and ownership come from these @ultrafuzz/runtime -// functions, loaded from the module the rendered workflow imports, rather than -// from copies kept in step with them. -const { isCredentialLikeEnvironmentVariableName, routeOwnsCredentialLikeEnvironmentVariable } = (await import( +// Credential classification, ownership and the built-in provider credentials +// come from @ultrafuzz/runtime, loaded from the module the rendered workflow +// imports, rather than from copies kept in step with them. +const { + BUILT_IN_PROVIDER_CREDENTIAL_ENVIRONMENT_VARIABLES, + isCredentialLikeEnvironmentVariableName, + routeOwnsCredentialLikeEnvironmentVariable +} = (await import( process.env.ULTRAFUZZ_RUNTIME_MODULE ?? new URL("../../modules/@ultrafuzz/runtime/dist/index.js", import.meta.url).href )) as { + BUILT_IN_PROVIDER_CREDENTIAL_ENVIRONMENT_VARIABLES: readonly string[]; isCredentialLikeEnvironmentVariableName: (name: string) => boolean; routeOwnsCredentialLikeEnvironmentVariable: (agent: string, name: string) => boolean; }; @@ -37,21 +42,6 @@ const CONTROLLER_ONLY_ENVIRONMENT_VARIABLES = [ "ULTRAFUZZ_WORKFLOW_PERSISTED_PATH" ] as const; -const BUILT_IN_PROVIDER_CREDENTIAL_ENVIRONMENT_VARIABLES = [ - "ANTHROPIC_API_KEY", - "ANTHROPIC_AUTH_TOKEN", - "AZURE_OPENAI_API_KEY", - // Claude Code's `claude setup-token` login; only the subscription Claude - // adapter puts it back (#1266). - "CLAUDE_CODE_OAUTH_TOKEN", - "CLAUDE_CODE_OAUTH_TOKEN_FILE_DESCRIPTOR", - "CODEX_API_KEY", - "DEEPSEEK_API_KEY", - "KIMI_API_KEY", - "MOONSHOT_API_KEY", - "OPENAI_API_KEY", - "OPENROUTER_API_KEY" -] as const; const BUILT_IN_PROVIDER_HOME_ENVIRONMENT_VARIABLES = [ "CLAUDE_CONFIG_DIR", "CODEX_HOME", diff --git a/packages/runtime/src/validate.ts b/packages/runtime/src/validate.ts index 3b5bf06e8..a3b7d7122 100644 --- a/packages/runtime/src/validate.ts +++ b/packages/runtime/src/validate.ts @@ -28,6 +28,7 @@ import { agentRegistryRegisters, inspectAgentRegistry } from "./agent-registry.j import { providerHomeDiagnostics, providerHomeProblems } from "./provider-home-preflight.js"; import { agentAuthenticationEnvironmentVariableNames, + BUILT_IN_PROVIDER_CREDENTIAL_ENVIRONMENT_VARIABLES, CLAUDE_SUBSCRIPTION_TOKEN_ENVIRONMENT_VARIABLE, isCredentialLikeEnvironmentVariableName, routeOwnsCredentialLikeEnvironmentVariable @@ -469,7 +470,12 @@ function blankedAllowlistedCredentialDiagnostics( for (const name of new Set((env.ULTRAFUZZ_AGENT_ENV_ALLOWLIST ?? "").split(",").map((entry) => entry.trim()))) { const upper = name.toUpperCase(); if (name === "" || !isCredentialLikeEnvironmentVariableName(upper) || consumed.has(upper)) continue; - if (selectedAgentRefs.some((agentRef) => routeOwnsCredentialLikeEnvironmentVariable(agentRef, upper))) continue; + // A route restores an allowlisted credential under its prefix, but never a + // built-in provider credential: only that provider's own configured + // authentication restores one, which `consumed` already covers. + const builtIn = (BUILT_IN_PROVIDER_CREDENTIAL_ENVIRONMENT_VARIABLES as readonly string[]).includes(upper); + if (!builtIn && selectedAgentRefs.some((agentRef) => routeOwnsCredentialLikeEnvironmentVariable(agentRef, upper))) + continue; diagnostics.push({ code: "AGENT_ENV_ALLOWLIST_CREDENTIAL_BLANKED", message: diff --git a/packages/runtime/test/provider-home-preflight.test.ts b/packages/runtime/test/provider-home-preflight.test.ts index 7d4f3bdf4..2cedc6c56 100644 --- a/packages/runtime/test/provider-home-preflight.test.ts +++ b/packages/runtime/test/provider-home-preflight.test.ts @@ -230,6 +230,28 @@ test("validate warns when an allowlisted credential reaches no selected agent (# // A subscription Claude agent consumes its own login; with no Codex agent, OPENAI_* reaches nobody. const claude = await validateProject({ projectRoot: project, env, agent: "ClaudeAgent" }); assert.deepEqual(blanked(claude).sort(), ["MY_SERVICE_TOKEN", "OPENAI_ORG_TOKEN"]); + + // A route never restores a built-in provider key from the allowlist, even + // under its own prefix: subscription Codex runs with OPENAI_API_KEY blank. + const configPath = path.join(project, "ultrafuzz.toml"); + const config = fs.readFileSync(configPath, "utf8"); + const subscriptionCodex = config.replace( + /\[agents\.CodexAgent\]\n(?:[^[]*\n)?/u, + '[agents.CodexAgent]\nauth = "subscription"\n\n' + ); + assert.notEqual(subscriptionCodex, config); + fs.writeFileSync(configPath, subscriptionCodex); + const codexEnv = { + ...env, + ULTRAFUZZ_AGENT_ENV_ALLOWLIST: `${env.ULTRAFUZZ_AGENT_ENV_ALLOWLIST},OPENAI_API_KEY`, + OPENAI_API_KEY: "codex-api-key" + }; + const subscription = await validateProject({ projectRoot: project, env: codexEnv, agent: "CodexAgent" }); + assert.deepEqual(blanked(subscription).sort(), ["CLAUDE_CODE_OAUTH_TOKEN", "MY_SERVICE_TOKEN", "OPENAI_API_KEY"]); + fs.writeFileSync(configPath, config); + // In api-key mode Codex consumes OPENAI_API_KEY itself. + const apiKey = await validateProject({ projectRoot: project, env: codexEnv, agent: "CodexAgent" }); + assert.deepEqual(blanked(apiKey).sort(), ["CLAUDE_CODE_OAUTH_TOKEN", "MY_SERVICE_TOKEN"]); }); // The prediction duplicates the adapters' rules, so check it against the adapter code itself.