Skip to content

fix(sdk): log the CodeAF missing-key warning once, and count the harness config's env - #1132

Merged
AbirAbbas merged 4 commits into
mainfrom
fix/sdk-codeaf-warning-once
Oct 27, 2026
Merged

AbirAbbas merged 4 commits into
mainfrom
fix/sdk-codeaf-warning-once

Conversation

@AbirAbbas

Copy link
Copy Markdown
Contributor

Summary

Follow-up to #1130. The v0.1.144 release sweep found that the CodeAF missing-key warning misfired in three ways, all in all three SDKs:

Setup v0.1.144 This PR
Agent with a harness config, no key, then refused calls warning logged twice (construction, then first refusal) once
Key passed only through the harness config's env map (calls work) false "harness calls will fail" warning at construction no warning
Process has a key, the harness config's env blanks it (calls are refused) no warning at construction warning

No call outcome changes: keyless calls are still refused before CodeAF is resolved, downloaded or launched, with the same HarnessProviderNotConfigured / ProviderUnavailableError{NotConfigured: true} error and message, and keyed calls are untouched. Only when the warning line is logged changes.

Root cause

  • fix(sdk): fail fast with a clear message when the CodeAF harness has no model key #1130's spec asked for two separate once-per-process budgets, one for Agent construction and one for call-time refusal, and its tests checked each path alone, never construction followed by a call. Each SDK therefore had two independent flags (Python _construction_warning_logged / _missing_key_logged, TS constructionWarningLogged / missingKeyLogged, Go agent.codeAFConfigWarning / harness.codeAFMissingKeyOnce).
  • The same spec said the construction check had no per-call environment to consult. That was wrong: the harness config's env map is applied to every call it makes (Python runner env, TS runner env, Go opts.Env = hc.Env). The construction check read only the process environment, while the call path read the process environment plus that map.

Changes

  • Shared budget: one warning flag per process, shared by construction and refusal (Go: one sync.Once in package harness, exposed as harness.WarnCodeAFMissingKeyOnce so the agent package prints through its configured logger).
  • Same environment as the call: the construction check builds its environment through the same function the call path uses (_child_environment / childEnvironment / CodeAFHasCredentials with Options{Env: cfg.HarnessConfig.Env}), with the config's env map as the overlay.
  • Contract text: the K2 and K6 sentences in sdk/codeaf-auth-contract.json now describe the shared budget and the effective environment. All 42 cases are unchanged.
  • Docs: docs/harness-providers.md and the agentfield skill snapshot say the same thing; the embedded mirror is kept byte-identical.

Validation contract

  • W1 One warning per process. Construction (no key) + two refused calls → 1; two constructions + one refused call → 1; no harness config + two refused calls → 1 (at the first refusal); construction with a key visible + a call whose env clears it → 1 (at the refusal).
  • W2 Every refused call still fails with the unchanged error: three refused calls → three errors, one warning.
  • W3 For every fixture case, with its overlay supplied as the harness config's env map, construction warns if and only if the case is not allowed.
  • W4 Keyed calls, legacy aforge pins, never-fail construction and refusal-before-launch are unchanged.
  • W5 Python, Go and TypeScript behave identically and keep the existing strings byte-for-byte.

New tests are named after the W item they prove, in test_harness_codeaf_auth.py, harness_codeaf_auth.test.ts, codeaf_auth_test.go and agent/harness_auth_test.go. The Go agent tests run each W1/W3 case in a fresh test process, so the shared sync.Once starts unspent without an exported reset.

Test plan

  • Python: ruff check . (0.15.22), mypy --config-file mypy.ini agentfield/, ./scripts/run_pytest.sh (full suite) — all clean
  • Go: go mod tidy (no change), go build ./..., go vet ./..., go test ./... — 8 packages ok
  • TypeScript: npm ci, npm run lint, npm test — 98 files, 1380 passed, 1 skipped
  • scripts/sync-embedded-skills.sh --check and control-plane TestSkillCatalogAndEmbeddedMirrorsStayAligned
  • Re-ran the release-sweep probe against the released v0.1.144 SDK and this branch, with an empty environment and fresh homes: two keyless calls after construction log 2 warnings before and 1 after (both calls still raise HarnessProviderNotConfigured, no network); a key only in HarnessConfig.env logs 1 false warning before and 0 after; a process key blanked by HarnessConfig.env logs 0 before and 1 after.

🤖 Generated with Claude Code

AbirAbbas and others added 4 commits October 27, 2026 00:08
The shared contract and the provider docs said a refused call warns once and
an Agent warns once at construction, so a keyless agent logged the same
warning twice. They now say the warning is logged once per process, at
construction when the effective environment (the process environment plus the
harness config's env map) has no key, otherwise at the first refused call.
Every refused call still fails with the not-configured error.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…arness config's env

Agent construction and refused calls kept separate once-flags, so a keyless
agent logged the warning twice. The construction check also read the process
environment only, so a key passed through HarnessConfig.env drew a false
"harness calls will fail" warning, and a config env that blanks the key drew
none. Both paths now share one flag, and construction builds the environment
through the same _child_environment call the harness calls use.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…nfig.Env

The agent package had its own sync.Once beside the harness package's, so a
keyless agent logged the warning at construction and again at the first
refused call, and the construction check ignored HarnessConfig.Env. One Once
in package harness now serves both paths through WarnCodeAFMissingKeyOnce, and
construction checks CodeAFHasCredentials with the config's Env.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…arnessConfig.env

Construction and refused calls kept separate flags, so a keyless agent warned
twice, and the construction check ignored harnessConfig.env. Both paths now
share one flag, and construction passes the config's env through the same
childEnvironment call the harness calls use.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@AbirAbbas
AbirAbbas requested a review from a team as a code owner October 27, 2026 04:08
@github-actions

Copy link
Copy Markdown
Contributor

Performance

SDK Memory Δ Latency Δ Tests Status
Python 9.1 KB - 0.33 µs -6% ✓ ✓
Go 220 B -21% 0.62 µs -38% ✓ ✓
TS 454 B +30% 1.91 µs -5% ✓ ⚠

⚠ Regression detected:

  • TypeScript memory: 350 B → 454 B (+30%)

@github-actions

Copy link
Copy Markdown
Contributor

📊 Coverage gate

Thresholds from .coverage-gate.toml: per-surface ≥ 84%, aggregate ≥ 85%, max per-surface regression ≤ 1.0 pp, max aggregate regression ≤ 0.50 pp.

Surface Current Baseline Δ
control-plane 88.20% 87.40% ↑ +0.80 pp 🟡
sdk-go 93.70% 92.00% ↑ +1.70 pp 🟢
sdk-python 94.72% 93.73% ↑ +0.99 pp 🟢
sdk-typescript 92.50% 90.42% ↑ +2.08 pp 🟢
web-ui 84.89% 84.79% ↑ +0.10 pp 🟡
aggregate 86.11% 85.75% ↑ +0.36 pp 🟡

✅ Gate passed

No surface regressed past the allowed threshold and the aggregate stayed above the floor.

@github-actions

Copy link
Copy Markdown
Contributor

📐 Patch coverage gate

Threshold: 80% on lines this PR touches vs origin/main (from .coverage-gate.toml:thresholds.min_patch).

Surface Touched lines Patch coverage Status
control-plane 0 — ➖ no changes
sdk-go 5 80.00% ✅
sdk-python 0 — ➖ no changes
sdk-typescript 8 100.00% ✅
web-ui 0 — ➖ no changes

✅ Patch gate passed

Every surface whose lines were touched by this PR has patch coverage at or above the threshold.

@AbirAbbas
AbirAbbas merged commit d26f52d into main Oct 27, 2026
47 checks passed
@AbirAbbas
AbirAbbas deleted the fix/sdk-codeaf-warning-once branch October 27, 2026 04:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant