Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@

### Other changes

- **[runtime] [docs]** A subscription-mode `ClaudeAgent` now runs on a headless host: `CLAUDE_CODE_OAUTH_TOKEN`, the long-lived login from `claude setup-token`, reaches Claude agents the way an API key reaches its own adapter, and every other model child has it blanked (#1266). Before, Ultrafuzz stripped it from every agent, even when allowlisted, and each Claude task failed the engine's preflight with "Claude Code is not logged in". `ultrafuzz validate` now warns with `AGENT_ENV_ALLOWLIST_CREDENTIAL_BLANKED` when `ULTRAFUZZ_AGENT_ENV_ALLOWLIST` names a credential that no selected agent uses, since every agent would receive it blank.
- **[runtime]** Production copies of `proxy-addr`, `vue`, `source-map-js` and `prosemirror-view` move past the Critical advisory GHSA-jqcg-44mw-7w3h (`proxy-addr` trusted every client as a proxy when a trust subnet was an IPv4-mapped IPv6 address with a short prefix) and the High advisories GHSA-g2v6-rqmx-r4w6 (XSS in `@vue/server-renderer`), GHSA-68fv-2mgg-jv7q (event-loop denial of service in `source-map-js`) and GHSA-c8x8-7fp4-3x9w (XSS in `prosemirror-view` paste handling). All four reach Ultrafuzz only through the workflow engine: Express under `@modelcontextprotocol/sdk`, and the Milkdown editor in `@smthrs/ui`. In `pnpm-lock.yaml`, `proxy-addr` moves from 2.0.7 to 2.0.8, `vue` and its `@vue/*` packages from 3.5.40 to 3.5.42, `source-map-js` from 1.2.1 to 1.2.2 and `prosemirror-view` from 1.42.2 to 1.42.3, all within their parents' ranges. Like any install that changes the engine's dependency tree, `pnpm install` of this change moves the engine to a new directory: a run still in progress in that checkout can fail until `ultrafuzz resume` continues it, and long-lived processes such as the dashboard need a restart. The engine that `run` installs from npm does not read that lockfile; it resolves every open range as of a fixed instant, which moves from `2026-08-18T06:00:00Z` to `2026-10-01T00:00:00Z`. Newly launched runs therefore also get the fixed releases (`vue` 3.5.43 and `prosemirror-view` 1.42.6 there), and `fast-uri` 3.1.8 and `undici` 7.30.0, which clear seven High advisories in that install that earlier lockfile fixes did not reach (GHSA-5jgf-p345-68v8, GHSA-f65p-4m7j-42xc, GHSA-fph4-wmhf-6fwf, GHSA-jqff-g426-hqxp and GHSA-qw65-cvwx-89v3 in `fast-uri`, GHSA-rfgv-xxqx-mfg5 and GHSA-w293-vg96-wgc3 in `undici`). The new instant also moves 94 other packages of the roughly 700 in that install to their newest in-range release as of that date, including `react` and `react-dom` from 19.2.8 to 19.3.0 and `ai` from 7.0.66 to 7.0.126.
- **[runtime]** The workspace copy of `@modelcontextprotocol/sdk` moves from 1.29.0 to 1.31.0, past the High advisory GHSA-6qxp-vccf-f47h (the SDK's OAuth client could send credentials to an authorization server chosen by the MCP server). The workflow engine that `run` installs already resolves 1.31.0 at its 2026-10-01 dependency cutoff.
- **[runtime] [docs]** `validate`, and so `run`, now refuse an agent provider home that the agent adapters would refuse, before planning, with `PROVIDER_HOME_UNSAFE` naming the directory and the command that fixes it, such as `chmod 700 ~/.codex` (#1265). `ultrafuzz doctor` reports the same diagnostic. Claude Code and Codex create `~/.claude` and `~/.codex` from the umask, usually `0755` or `0775`, and the adapters accept only `0700` for a provider home and for the Ultrafuzz provider-home root above one. So any Codex user (the `init` default) on macOS or Linux, and any Claude user on Linux, whose CLI had created its home failed only once the engine rendered the workflow, after about 2 minutes of planning and execution-snapshot preparation, with an engine `INTERNAL_ERROR` (`provider homes must be operator-owned with private 0700 permissions`) that named no directory. The check covers every agent with a provider home, `OpenRouterAgent` included, and resolves it as the adapters do, from the launch environment's `HOME` or, when that environment has no `HOME`, from the operator's home directory. It also reports a relative `ULTRAFUZZ_PROVIDER_HOME_ROOT`, `CODEX_HOME`, `CLAUDE_CONFIG_DIR`, `KIMI_CODE_HOME` or `KIMI_SHARE_DIR`, which the adapters refuse, and a directory it cannot inspect; a directory you cannot search is reported with `chmod u+x` on that directory, and the printed command quotes a path that needs it. Ultrafuzz never changes these directories itself.
Expand Down
8 changes: 8 additions & 0 deletions docs/config.md
Original file line number Diff line number Diff line change
Expand Up @@ -140,6 +140,14 @@ for running multiple Claude subscriptions side by side. Set `auth = "api-key"`
to bill against the Anthropic API using canonical `ANTHROPIC_API_KEY`. Both modes drive the `claude` CLI, so it must be
installed either way; `auth` only changes how that CLI authenticates.

On a headless host such as a container or CI runner, where `claude auth login`
cannot open a browser, generate a long-lived token with `claude setup-token`
on any machine and export it as `CLAUDE_CODE_OAUTH_TOKEN` before `ultrafuzz
run`. In subscription mode `ClaudeAgent` receives it, with no
`ULTRAFUZZ_AGENT_ENV_ALLOWLIST` entry, and every other agent has it blanked.
Export it again before `ultrafuzz resume`. `ClaudeAgent` in `api-key` mode does
not receive it.

When a Claude profile sets `reasoning`, the generated adapter passes it to the
current Claude Code CLI as `--effort <value>`. Pin the model through the
profile's `model` field rather than under `[agents.ClaudeAgent]`.
Expand Down
5 changes: 4 additions & 1 deletion docs/security.md
Original file line number Diff line number Diff line change
Expand Up @@ -253,6 +253,9 @@ children and omitted from unrelated Modal task secrets. A recognized
provider-route prefix such as `AWS_` scopes the value to that route. Arbitrary
unrecognized credentials are not supported and reach no model task. Stock
agent API keys use their validated canonical `api_key_env`; the active adapter
restores only its own key. Use only a credential-free RPC endpoint when an RPC
restores only its own key. A subscription `ClaudeAgent` likewise restores only
`CLAUDE_CODE_OAUTH_TOKEN`, Claude Code's headless login. `ultrafuzz validate`
warns when an allowlisted credential is owned by no selected agent, since every
agent would receive it blank. Use only a credential-free RPC endpoint when an RPC
URL must remain a global input.
Do not add unrelated credentials merely to make them available to prompts.
45 changes: 45 additions & 0 deletions packages/runtime/src/provider-credential-environment.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,51 @@ const ROUTE_ENV_PREFIXES: Readonly<Record<string, readonly string[]>> = {
KimiAgent: ["KIMI_", "MOONSHOT_"]
};

/**
* Provider credentials every model child has blanked. Only the adapter that
* owns one restores it, from its configured authentication; an
* ULTRAFUZZ_AGENT_ENV_ALLOWLIST entry never does.
*/
export const BUILT_IN_PROVIDER_CREDENTIAL_ENVIRONMENT_VARIABLES = [
"ANTHROPIC_API_KEY",
"ANTHROPIC_AUTH_TOKEN",
"AZURE_OPENAI_API_KEY",
// Claude Code's `claude setup-token` login; only the subscription Claude
// adapter puts it back (#1266).
"CLAUDE_CODE_OAUTH_TOKEN",
"CLAUDE_CODE_OAUTH_TOKEN_FILE_DESCRIPTOR",
"CODEX_API_KEY",
"DEEPSEEK_API_KEY",
"KIMI_API_KEY",
"MOONSHOT_API_KEY",
"OPENAI_API_KEY",
"OPENROUTER_API_KEY"
] as const;

/**
* Claude Code's documented non-interactive login: a long-lived token from
* `claude setup-token`. A subscription-mode ClaudeAgent receives it the way an
* api-key agent receives its key, and every other model child has it blanked
* (#1266).
*/
export const CLAUDE_SUBSCRIPTION_TOKEN_ENVIRONMENT_VARIABLE = "CLAUDE_CODE_OAUTH_TOKEN";

/** The credential variables an agent's adapter reads for its configured authentication. */
export function agentAuthenticationEnvironmentVariableNames(
agentRef: string,
agent: { auth?: string; apiKeyEnv?: string }
): string[] {
if (agent.auth === "api-key" && agent.apiKeyEnv !== undefined) {
return agentRef === "KimiAgent" && agent.apiKeyEnv === "KIMI_API_KEY"
? [agent.apiKeyEnv, "MOONSHOT_API_KEY"]
: [agent.apiKeyEnv];
}
if (agentRef === "ClaudeAgent" && agent.auth === "subscription") {
return [CLAUDE_SUBSCRIPTION_TOKEN_ENVIRONMENT_VARIABLE];
}
return [];
}

export function isCredentialLikeEnvironmentVariableName(name: string): boolean {
return isSensitiveEnvironmentName(name);
}
Expand Down
17 changes: 9 additions & 8 deletions packages/runtime/src/start-run.ts
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,7 @@ import {
type WorkflowLifecycleValue
} from "./types.js";
import { planRun } from "./plan-run.js";
import { agentAuthenticationEnvironmentVariableNames } from "./provider-credential-environment.js";
import { refreshRunPrompts } from "./prompt-refresh.js";
import { probeCommandsForExecution } from "./required-commands.js";
import { forgeGuardMetadata, prepareForgeGuardEnvironment } from "./forge-guard.js";
Expand Down Expand Up @@ -2047,10 +2048,11 @@ function agentCredentialEnvironmentVariableNames(config: ResolvedConfig, agentRe
const activeAgentRefs = new Set(agentRefs);
const names: string[] = [];
for (const [agentRef, agent] of Object.entries(config.agents)) {
if (!activeAgentRefs.has(agentRef) || agent.auth !== "api-key" || agent.apiKeyEnv === undefined) continue;
assertCredentialEnvironmentVariableName(agent.apiKeyEnv);
names.push(agent.apiKeyEnv);
if (agentRef === "KimiAgent" && agent.apiKeyEnv === "KIMI_API_KEY") names.push("MOONSHOT_API_KEY");
if (!activeAgentRefs.has(agentRef)) continue;
for (const name of agentAuthenticationEnvironmentVariableNames(agentRef, agent)) {
assertCredentialEnvironmentVariableName(name);
names.push(name);
}
}
return [...new Set(names)].sort();
}
Expand All @@ -2065,10 +2067,9 @@ function agentEnvironmentVariableNames(
if (activeAgentRefs.size > 0) names.push("ULTRAFUZZ_PROVIDER_HOME_ROOT", ...MODEL_ROUTE_PROXY_ENVIRONMENT_VARIABLES);
for (const [agentRef, agent] of Object.entries(config.agents)) {
if (!activeAgentRefs.has(agentRef)) continue;
if (agent.auth === "api-key" && agent.apiKeyEnv !== undefined) {
assertCredentialEnvironmentVariableName(agent.apiKeyEnv);
names.push(agent.apiKeyEnv);
if (agentRef === "KimiAgent" && agent.apiKeyEnv === "KIMI_API_KEY") names.push("MOONSHOT_API_KEY");
for (const name of agentAuthenticationEnvironmentVariableNames(agentRef, agent)) {
assertCredentialEnvironmentVariableName(name);
names.push(name);
}
if (agentRef === "KimiAgent") {
names.push("KIMI_BASE_URL", "KIMI_CODE_HOME", "KIMI_SHARE_DIR");
Expand Down
20 changes: 18 additions & 2 deletions packages/runtime/src/templates/smithers/agents/claude.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -57,9 +57,17 @@ export class CompatibleClaudeCodeAgent extends StatedFailureClaudeCodeAgent {
this.opts.settingSources = "user";
const command = await super.buildCommand(params);
try {
// The command environment is applied last when Smithers spawns, so the
// subscription login set on this agent (#1266) has to be carried into it,
// or the blanked provider credential would override it.
const login = this.opts.env?.CLAUDE_CODE_OAUTH_TOKEN;
return {
...command,
env: workflowControlChildEnvironment(command.env, process.env, { agent: "ClaudeAgent" })
env: workflowControlChildEnvironment(
login === undefined || login === "" ? command.env : { CLAUDE_CODE_OAUTH_TOKEN: login, ...command.env },
process.env,
{ agent: "ClaudeAgent" }
)
};
} catch (error) {
await command.cleanup?.();
Expand Down Expand Up @@ -100,7 +108,15 @@ function claudeAuthOptions(): ClaudeAuthOptions {
// ClaudeCodeAgent clears ANTHROPIC_API_KEY so the logged-in subscription (`claude -p`)
// is used. configDir is always forwarded; without config_dir or ULTRAFUZZ_PROVIDER_HOME_ROOT,
// resolveProviderHome falls back to $CLAUDE_CONFIG_DIR, then ~/.claude.
return { configDir };
// On a headless host the subscription is a `claude setup-token` token instead of a login
// in the provider home. Every other model child has it blanked (#1266).
const token = process.env.CLAUDE_CODE_OAUTH_TOKEN;
return token === undefined || token === ""
? { configDir }
: {
configDir,
env: { CLAUDE_CODE_OAUTH_TOKEN: workflowControlCredentialValue(token, "CLAUDE_CODE_OAUTH_TOKEN") }
};
}
throw new Error(`unsupported ClaudeAgent auth mode in ultrafuzz.toml: ${auth}`);
}
Expand Down
24 changes: 9 additions & 15 deletions packages/runtime/src/templates/smithers/agents/environment.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -4,13 +4,18 @@ import { fileURLToPath } from "node:url";
export const PROVIDER_SCOPED_SENSITIVE_ENVIRONMENT_CAPABILITY =
"ultrafuzz.provider-scoped-sensitive-environment.v1" as const;

// Credential classification and ownership come from these @ultrafuzz/runtime
// functions, loaded from the module the rendered workflow imports, rather than
// from copies kept in step with them.
const { isCredentialLikeEnvironmentVariableName, routeOwnsCredentialLikeEnvironmentVariable } = (await import(
// Credential classification, ownership and the built-in provider credentials
// come from @ultrafuzz/runtime, loaded from the module the rendered workflow
// imports, rather than from copies kept in step with them.
const {
BUILT_IN_PROVIDER_CREDENTIAL_ENVIRONMENT_VARIABLES,
isCredentialLikeEnvironmentVariableName,
routeOwnsCredentialLikeEnvironmentVariable
} = (await import(
process.env.ULTRAFUZZ_RUNTIME_MODULE ??
new URL("../../modules/@ultrafuzz/runtime/dist/index.js", import.meta.url).href
)) as {
BUILT_IN_PROVIDER_CREDENTIAL_ENVIRONMENT_VARIABLES: readonly string[];
isCredentialLikeEnvironmentVariableName: (name: string) => boolean;
routeOwnsCredentialLikeEnvironmentVariable: (agent: string, name: string) => boolean;
};
Expand All @@ -37,17 +42,6 @@ const CONTROLLER_ONLY_ENVIRONMENT_VARIABLES = [
"ULTRAFUZZ_WORKFLOW_PERSISTED_PATH"
] as const;

const BUILT_IN_PROVIDER_CREDENTIAL_ENVIRONMENT_VARIABLES = [
"ANTHROPIC_API_KEY",
"ANTHROPIC_AUTH_TOKEN",
"AZURE_OPENAI_API_KEY",
"CODEX_API_KEY",
"DEEPSEEK_API_KEY",
"KIMI_API_KEY",
"MOONSHOT_API_KEY",
"OPENAI_API_KEY",
"OPENROUTER_API_KEY"
] as const;
const BUILT_IN_PROVIDER_HOME_ENVIRONMENT_VARIABLES = [
"CLAUDE_CONFIG_DIR",
"CODEX_HOME",
Expand Down
52 changes: 51 additions & 1 deletion packages/runtime/src/validate.ts
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,13 @@ import { effectiveAuditPolicy } from "./audit-profile-policy.js";
import { timeoutShadowingDiagnostics } from "./timeout-shadowing.js";
import { agentRegistryRegisters, inspectAgentRegistry } from "./agent-registry.js";
import { providerHomeDiagnostics, providerHomeProblems } from "./provider-home-preflight.js";
import {
agentAuthenticationEnvironmentVariableNames,
BUILT_IN_PROVIDER_CREDENTIAL_ENVIRONMENT_VARIABLES,
CLAUDE_SUBSCRIPTION_TOKEN_ENVIRONMENT_VARIABLE,
isCredentialLikeEnvironmentVariableName,
routeOwnsCredentialLikeEnvironmentVariable
} from "./provider-credential-environment.js";
import { promptTextsForCatalog, transformPromptCatalogForRun, transformTopologyForRun } from "./topology-transform.js";
import {
configDiagnostics,
Expand Down Expand Up @@ -372,6 +379,7 @@ function evaluatePolicies(
// The adapters refuse an unsafe provider home only once the engine renders the workflow, after
// planning and the execution snapshot (#1265). Refusing it here fails `run` before either.
const providerHomes = providerHomeDiagnostics(providerHomeProblems(selectedAgentRefs, config, env));
const blankedCredentials = blankedAllowlistedCredentialDiagnostics(selectedAgentRefs, config, env);
return {
posture: {
paths: postureFromDiagnostics("paths", "product files are written through project-local path guards", []),
Expand All @@ -380,7 +388,7 @@ function evaluatePolicies(
providerHomes.length === 0
? "agent references resolve before launch"
: "agent references resolve, but an agent's provider home would be refused at launch",
[...agentRegistry, ...providerHomes]
[...agentRegistry, ...providerHomes, ...blankedCredentials]
),
trust: postureFromDiagnostics(
"trust",
Expand Down Expand Up @@ -440,3 +448,45 @@ function applyAgentOverrides(config: ResolvedConfig, input: ValidateProjectInput
reasoning: input.reasoning
});
}

/**
* An allowlisted variable that looks like a credential reaches only the agent
* route that owns it. When no selected agent owns it, every agent receives it
* blank, which surfaces only as that agent's own authentication failure. Say
* so before launch (#1266).
*/
function blankedAllowlistedCredentialDiagnostics(
selectedAgentRefs: readonly string[],
config: ResolvedConfig,
env: Record<string, string | undefined>
): RuntimeDiagnostic[] {
const consumed = new Set(
selectedAgentRefs.flatMap((agentRef) => {
const agent = config.agents[agentRef];
return agent === undefined ? [] : agentAuthenticationEnvironmentVariableNames(agentRef, agent);
})
);
const diagnostics: RuntimeDiagnostic[] = [];
for (const name of new Set((env.ULTRAFUZZ_AGENT_ENV_ALLOWLIST ?? "").split(",").map((entry) => entry.trim()))) {
const upper = name.toUpperCase();
if (name === "" || !isCredentialLikeEnvironmentVariableName(upper) || consumed.has(upper)) continue;
// A route restores an allowlisted credential under its prefix, but never a
// built-in provider credential: only that provider's own configured
// authentication restores one, which `consumed` already covers.
const builtIn = (BUILT_IN_PROVIDER_CREDENTIAL_ENVIRONMENT_VARIABLES as readonly string[]).includes(upper);
if (!builtIn && selectedAgentRefs.some((agentRef) => routeOwnsCredentialLikeEnvironmentVariable(agentRef, upper)))
continue;
diagnostics.push({
code: "AGENT_ENV_ALLOWLIST_CREDENTIAL_BLANKED",
message:
`${name} is in ULTRAFUZZ_AGENT_ENV_ALLOWLIST but looks like a credential that no selected agent uses, ` +
"so every agent receives it blank" +
(upper === CLAUDE_SUBSCRIPTION_TOKEN_ENVIRONMENT_VARIABLE
? `; ClaudeAgent with auth = "subscription" receives ${CLAUDE_SUBSCRIPTION_TOKEN_ENVIRONMENT_VARIABLE} without the allowlist`
: ""),
severity: "warning",
source: "agents"
});
}
return diagnostics;
}
Loading
Loading