Skip to content

Make CodeAF the default harness, installed automatically with AgentField - #1128

Merged
AbirAbbas merged 42 commits into
mainfrom
feat/codeaf-default-harness
Oct 24, 2026
Merged

AbirAbbas merged 42 commits into
mainfrom
feat/codeaf-default-harness

Conversation

@AbirAbbas

@AbirAbbas AbirAbbas commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

Summary

CodeAF (the renamed aforge-v2, Agent-Field/CodeAF) becomes the default harness in the Python, Go and TypeScript SDKs, and it is installed automatically wherever AgentField is. A provider-less app.harness(...) call runs codeaf exec --json headless with nothing else to set up.

  • New codeaf provider in all three SDKs. It runs codeaf exec --json --dir <root> with the prompt on stdin (AGENTFIELD_CODEAF_COMMAND=do opts into codeaf do). It reads CodeAF's current envelope (ok, stop, answer, spend_usd, tokens, model) and exit codes rather than the legacy aforge fields CodeAF keeps for one more release.

  • Nothing a user already chose changes.

    • provider="aforge" (or AGENTFIELD_HARNESS_PROVIDER=aforge) is an alias that runs CodeAF.
    • A legacy pin keeps the original aforge adapter and binary, both with provider="aforge" and when no provider is named (as on main). The pins are AFORGE_BIN, Python aforge_bin, TS aforgeBin, and Go Options.BinPath with the default provider.
    • AGENTFIELD_AFORGE_COMMAND and AFORGE_MODEL are honored as fallbacks for AGENTFIELD_CODEAF_COMMAND and CODEAF_MODEL.
  • Pinned, checksum-verified binary. CodeAF v0.8.2 is pinned, with all six release assets' SHA-256 embedded in each SDK and in af. It installs only into $AGENTFIELD_HOME/bin/codeaf and never touches a user's own CodeAF.

  • Installed by every path that installs AgentField:

    • install.sh / install.ps1
    • Desktop launch
    • af skill install --all
    • a successful af install
    • the python-agent, go-agent and control-plane-cloud images
    • af codeaf ensure, the explicit command
    • for pip/npm/Go-only users, a lazy download on the first harness call

    The installers are served from main but install the latest stable af. When that af has no codeaf command yet, they provision aforge exactly as before, so there is no gap between merging this and the next stable release. Opt out with --no-codeaf, -NoCodeaf or AGENTFIELD_SKIP_CODEAF=1; the old aforge opt-outs still work.

  • Quiet and isolated in automation.

    • CodeAF state lives in $AGENTFIELD_HOME/codeaf.
    • The update check is off.
    • Telemetry is off under CI, DO_NOT_TRACK or AGENTFIELD_TELEMETRY_ENABLED=false.
    • The default worker model is ~deepseek/deepseek-v4-flash-latest.

Behavior (validation contract)

  • With no provider named, CodeAF runs and usage is attributed harness=codeaf. An explicit provider or AGENTFIELD_HARNESS_PROVIDER still wins, and a blank value is ignored.
  • aforge without a pin runs CodeAF and logs one line. With a pin, it runs the legacy adapter unchanged.
  • Binary lookup order: explicit option, then CODEAF_BIN, then the managed binary, then a pinned download, then PATH, then a legacy aforge binary (with a warning). If none is found, an "unavailable" error suggests af codeaf ensure and includes the download error.
    • Failed lookups are not cached, and a download is retried at most once a minute.
    • The first-use download counts against the harness timeout. Explicit binaries never wait on it.
  • A download is SHA-256-verified before an atomic rename, and the version marker is written last, so a mismatch leaves nothing installed. An unwritable $AGENTFIELD_HOME/bin is reported as a download failure.
  • Exit and stop mapping:
    • done → success
    • deadline → timeout
    • unchecked, budget or turn-cap with an answer → success, logged as partial
    • question, incomplete, error or price → failure carrying CodeAF's own error text
  • The doctors (af doctor, af harness doctor and the SDK doctors) use the execution lookup order and never download. For aforge, they report what would actually run.

Rollback and compatibility

  • Set AFORGE_BIN=<path> to pin the old harness per process.
  • The aforge stage stays in the agent and cloud images.
  • af aforge ensure and its download mirror are unchanged.
  • Products (pr-af, sec-af, cloudsecurity-af, contract-af, SWE-AF, video-essay-af) select provider="aforge" without a pin, so after they pick up this SDK they run CodeAF through the alias. Follow-up PRs in each repo raise the SDK floor and ship CodeAF in their images.

Testing

Every run below uses the real CodeAF v0.8.2 and aforge v0.1.0 binaries. The model was a local OpenAI-compatible endpoint (Ollama, qwen3:14b, via CODEAF_BASE_URL), so no paid model was involved.

  • Zero-config, fresh home, Python, Go and TypeScript (packed npm tarball): first call downloads and verifies CodeAF, the model writes the requested file, and schema output is parsed.
  • Windows (native): the Python SDK downloads codeaf.exe and completes the same run.
  • Control plane + agent nodes (Python and Go): each reasoner runs app.harness() / agent.Harness() through the control plane, the result comes back through the execution API, and execution_usage records harness=codeaf with token counts.
  • Provider and env matrix (all three SDKs):
    • the alias, via argument and via env
    • the explicit legacy pin, and the implicit default with AFORGE_BIN (both run aforge v0.1.0)
    • AFORGE_MODEL alone selects the model; AGENTFIELD_AFORGE_COMMAND=do launches codeaf do with all seats pinned
    • an existing aforge user upgrading, online (gets CodeAF) and with the download blocked (falls back to aforge)
    • the opt-out with nothing installed, an unwritable home, HTTPS_PROXY set for TypeScript, a missing CODEAF_BIN
    • three concurrent calls (one download), and a forced timeout (exit 3 → timeout)
  • CLI:
    • af codeaf ensure: concurrent installs (one download), idempotent, --force, opt-out
    • af harness doctor / af doctor
    • the skill-install hook and its opt-out
    • af aforge ensure unchanged
  • Installers:
    • install.sh with stable af v0.1.142 → installs aforge; with this branch's af → installs CodeAF; --no-codeaf → installs neither
    • install.ps1 on Windows PowerShell 5.1 with main's and this branch's af.exe: both paths work, the caller's session env is untouched, and the skip variables are honored
  • Docker: the CodeAF stage script runs with empty (classic builder), amd64 and arm64 TARGETARCH; the cloud image is built and smoke-tested in CI.
  • CI gates, run locally with CI's literal steps on the final tree:
    • Python: ruff 0.15.22, mypy, full pytest on 3.11 and 3.10 (2,804 / 2,803 passed)
    • TypeScript: lint, full vitest (1,256 passed)
    • Go SDK: tidy, build, -race on the new resolver tests
    • control plane: build, linux/darwin/windows compile matrix
    • desktop: typecheck, tests (673 passed)
    • coverage gate passed; patch coverage: control plane 93%, Go SDK 96%, TypeScript 98%, Python changes 99%+ (measured separately; the gate's tracked-module list excludes the harness)
  • Three independent adversarial reviews of the landing tree. The first found two blockers (legacy pins and legacy command/model settings being dropped) and four should-fixes. The second found four smaller edge cases. All of them are fixed with tests from the contract above, and the final review's verdict was merge.

Not covered or not fully covered:

  • No paid OpenRouter call. The default model is reached the same way through CodeAF; only the endpoint differed.
  • codeaf do: launched correctly from the SDK, but a 14B local model does not converge on do within ten minutes, so it was not run to completion.
  • macOS: cross-compiled only.
  • python-agent and go-agent images: not built end to end (no Docker on the test machine). Their CodeAF stage script was run directly.
  • Unrelated failures seen:
    • TestDispatchAgentRequestStopsWaitingOnceTheNodeIsDemoted fails intermittently on main too (2 of 10 runs).
    • Python agents' harness usage is not stored for successful control-plane executions, sync or async; only failed ones get a usage row. main's SDK behaves identically (tested with a fake harness binary). The status handler returns early on an already-terminal execution before ingesting usage, which is the likely cause. Go agents are not affected. Worth a separate fix.

🤖 Generated with Claude Code

AbirAbbas and others added 23 commits October 23, 2026 16:22
…ed install

CodeafProvider runs `codeaf exec --json --dir <root>` (or `do` via
AGENTFIELD_CODEAF_COMMAND), parses CodeAF's current envelope and exit
ladder, and isolates its state under $AGENTFIELD_HOME/codeaf. The binary is
resolved lazily: explicit override, CODEAF_BIN, the managed
$AGENTFIELD_HOME/bin/codeaf, a checksum-pinned download of CodeAF v0.8.2,
PATH, and finally a legacy aforge binary.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…alias

Provider-less calls now run CodeAF. `provider="aforge"` keeps working and
runs CodeAF too, unless a legacy binary is pinned with AFORGE_BIN or
aforge_bin, which keeps the original aforge adapter. Harness results carry
the provider that actually ran so usage is attributed correctly.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…anaged install

CodeafProvider runs CodeAF headless (`exec` by default, `do` via
AGENTFIELD_CODEAF_COMMAND), parses the current envelope and exit ladder,
and resolves the binary lazily, downloading the checksum-pinned v0.8.2
release into $AGENTFIELD_HOME/bin when it is missing.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
… an alias

Provider-less calls run CodeAF. 'aforge' still works and runs CodeAF
unless aforgeBin or AFORGE_BIN pins the legacy binary. Usage is attributed
to the provider that actually ran.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…nstall

ProviderCodeAF runs CodeAF headless (`exec` by default, `do` via
AGENTFIELD_CODEAF_COMMAND), maps its current envelope and exit ladder onto
harness results, and resolves the binary lazily, downloading the
checksum-pinned v0.8.2 release into $AGENTFIELD_HOME/bin when missing.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
DefaultProvider is now ProviderCodeAF. ProviderAforge still works and runs
CodeAF unless Options.BinPath or AFORGE_BIN pins the legacy binary, which
keeps the original aforge adapter. Usage follows the provider that ran.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
internal/codeaf installs the CodeAF v0.8.2 release binary into
$AGENTFIELD_HOME/bin, verified against embedded SHA-256 pins, with a lock,
a version marker and the AGENTFIELD_SKIP_CODEAF / AGENTFIELD_SKIP_AFORGE
opt-outs. `af aforge ensure` is unchanged. Both doctors gain a codeaf row.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…l and af run

The skill catalog install now provisions CodeAF instead of aforge, and
`af install` / `af run` repair a missing CodeAF best-effort, so upgrading
af alone is enough to get the default harness.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
install.sh / install.ps1 provision CodeAF (opt out with --no-codeaf /
-NoCodeaf; the old aforge opt-outs still work). The python, go and cloud
images bake the pinned CodeAF binary and fail the build if it cannot be
verified; the aforge stage stays as a rollback hatch.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…e homes and proxies

A failed lookup or download is no longer cached for the life of the
process: later calls re-check every location and retry the download at most
once a minute. CodeAF's state falls back to a private temp directory when
$AGENTFIELD_HOME is read-only, the doctor honours CODEAF_BIN, and downloads
go through HTTP(S)_PROXY with NO_PROXY support.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…empty env values

Failed lookups and downloads are no longer cached for the life of the
process; later calls re-check every location and retry the download at most
once a minute. CodeAF's state falls back to a private temp directory when
$AGENTFIELD_HOME is read-only, and an explicitly empty per-call variable
(such as CODEAF_TELEMETRY_ENDPOINT="") now reaches the CodeAF child instead
of being dropped. Legacy adapters keep their existing env handling.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
… homes

Failed lookups and downloads are no longer cached for the life of the
process; later calls re-check every location and retry the download at most
once a minute, and a truncated download now falls through to PATH and the
legacy binary instead of raising. CodeAF's state falls back to a private
temp directory when $AGENTFIELD_HOME is read-only, and the doctor honours
CODEAF_BIN and the temp-managed binary in the same order as execution.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…install

`af run` no longer waits: when CodeAF is missing it starts a detached
`af codeaf ensure` in the background and launches the agent immediately.
`af install` and skill installs honour the command context and give up on a
busy install lock after 30 s. Downloads go through HTTP(S)_PROXY, and the
doctors find CODEAF_BIN and the managed binaries in the same order as
execution. `af codeaf ensure --bin-dir` installs beside a custom af.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
… and custom installs

The installers resolve a single managed home for the whole run (staging,
explicit AGENTFIELD_HOME, or ~/.agentfield) and pass it to every child, so
a staging install no longer downloads CodeAF twice, and a custom install
directory gets CodeAF beside af.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…ecked

The temp-directory fallbacks for the binary and its state now live under
<tmp>/agentfield-codeaf-<uid> and are only used when the directory is a real
directory owned by the current user with mode 0700, so another local user
cannot plant a binary or share the state directory.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The temp-directory fallbacks for the binary and its state now live under
<tmp>/agentfield-codeaf-<uid> and are only used when the directory is a real
directory owned by the current user with mode 0700, so another local user
cannot plant a binary or share the state directory.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The temp-directory fallbacks for the binary and its state now live under
<tmp>/agentfield-codeaf-<uid> and are only used when the directory is a real
directory owned by the current user with mode 0700. An existing shared or
world-writable directory is no longer accepted.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…per-user temp copies

`af run` now takes a cross-process lease before starting the background
`af codeaf ensure`, so many launches while a download is in flight start
one installer, and a failed attempt is not retried automatically for ten
minutes. The child is reaped instead of released. The doctors read the
temp fallback only from an owner-checked per-user directory, and
`--bin-dir` is gone: CodeAF always installs into $AGENTFIELD_HOME/bin.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Custom install directories no longer get a second copy beside af; the
skill hook and the explicit ensure now target the same file, so a fresh
install downloads CodeAF exactly once.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…before reading stdin

A provider CLI that fails at startup (a missing key, a bad flag) can close
its stdin before the prompt is written. The write error was unhandled and
surfaced as an uncaught exception in the agent process; the exit code and
stderr already report the failure, so the stream error is now ignored.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@AbirAbbas
AbirAbbas requested a review from a team as a code owner October 23, 2026 21:18
response.end(bytes);
}
if (kind === 'http') response.writeHead(404).end();
if (kind === 'redirect-loop') response.writeHead(302, { location: request.url }).end();
@github-actions

github-actions Bot commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

Performance

SDK Memory Δ Latency Δ Tests Status
Python 9.1 KB - 0.26 µs -26% ✓ ✓
Go 232 B -17% 0.68 µs -32% ✓ ✓
TS 410 B +17% 1.92 µs -4% ✓ ⚠

⚠ Regression detected:

  • TypeScript memory: 350 B → 410 B (+17%)

@github-actions

github-actions Bot commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

📊 Coverage gate

Thresholds from .coverage-gate.toml: per-surface ≥ 84%, aggregate ≥ 85%, max per-surface regression ≤ 1.0 pp, max aggregate regression ≤ 0.50 pp.

Surface Current Baseline Δ
control-plane 88.20% 87.40% ↑ +0.80 pp 🟡
sdk-go 93.60% 92.00% ↑ +1.60 pp 🟢
sdk-python 94.72% 93.73% ↑ +0.99 pp 🟢
sdk-typescript 92.37% 90.42% ↑ +1.95 pp 🟢
web-ui 84.89% 84.79% ↑ +0.10 pp 🟡
aggregate 86.11% 85.75% ↑ +0.36 pp 🟡

✅ Gate passed

No surface regressed past the allowed threshold and the aggregate stayed above the floor.

@github-actions

github-actions Bot commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

📐 Patch coverage gate

Threshold: 80% on lines this PR touches vs origin/main (from .coverage-gate.toml:thresholds.min_patch).

Surface Touched lines Patch coverage Status
control-plane 334 93.00% ✅
sdk-go 568 96.00% ✅
sdk-python 0 — ➖ no changes
sdk-typescript 291 98.00% ✅
web-ui 0 — ➖ no changes

✅ Patch gate passed

Every surface whose lines were touched by this PR has patch coverage at or above the threshold.

@AbirAbbas AbirAbbas closed this Oct 23, 2026
@AbirAbbas AbirAbbas reopened this Oct 23, 2026
@AbirAbbas AbirAbbas closed this Oct 23, 2026
@AbirAbbas AbirAbbas reopened this Oct 23, 2026
@santoshkumarradha

Copy link
Copy Markdown
Member

@AbirAbbas can we do a round of checks for sweaf and praf as well ? 🙏🏻

AbirAbbas and others added 19 commits October 23, 2026 22:10
install.sh and install.ps1 are served from main but install the latest
stable af. Until that release has `af codeaf`, `af codeaf ensure`
fails and the machine ended up with no harness at all, while the
released SDKs still default to aforge. Detect the command in `af --help`
and, when it is missing, provision aforge exactly as earlier installers
did.

install.ps1 also stops changing AGENTFIELD_HOME and the skip variables
in the caller's session (iwr | iex runs in that scope), keeps main's
install directory, and drops the -Staging switch it never had before.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The classic (non-BuildKit) builder leaves TARGETARCH unset, which made
the fail-closed CodeAF stage exit with "Unsupported architecture".
Fall back to dpkg --print-architecture.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
af run no longer spawns a detached `af codeaf ensure` (and its lease and
failure cooldown files); the curl installer, skill install, `af install`,
Desktop and the SDKs' own lazy download already provision CodeAF, so
af run is byte-identical to main again. The CLI lookup and doctors also
stop consulting the per-user temp directory, which the SDKs no longer
use.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Remove the per-user temp-directory fallback. An unwritable home bin
directory is now an ordinary download failure (reported in the
unavailable error, retried at most once a minute), and CODEAF_HOME is
created best-effort without a temp fallback.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Same change as the Python SDK: no temp-directory fallback for the binary
or for CODEAF_HOME. Without any home directory, CODEAF_HOME is left to
CodeAF's own default instead of a cwd-relative path.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…fallback

The downloader is a plain https.get with redirects again. When
HTTPS_PROXY is set the SDK skips the download (a direct attempt would
hang) and points at `af codeaf ensure`, which honors proxies. The
per-user temp-directory fallback is removed as in the other SDKs.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
An older af's `aforge ensure` fallback only knows AGENTFIELD_SKIP_AFORGE,
so AGENTFIELD_SKIP_CODEAF=1 still downloaded aforge there. Treat either
skip variable as CODEAF_MODE=none up front, as install.sh already does.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
… the timeout

- No provider named + a legacy pin (AFORGE_BIN, aforge_bin) keeps the
  legacy aforge adapter, as on main, instead of switching to CodeAF.
- AGENTFIELD_AFORGE_COMMAND and AFORGE_MODEL are honored when the CodeAF
  variables are unset (per-call env first), so the alias no longer
  replaces a chosen command or model with the defaults.
- The harness timeout now covers waiting for the first-use download; a
  caller that runs out of time gets a timeout result while the shared
  download keeps going for later calls. Execution gets the remaining
  budget.
- harness_doctor reports what provider 'aforge' would actually run.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…across downloads

Same compatibility rules as the Python SDK (implicit default + AFORGE_BIN
or Options.BinPath keeps the aforge adapter; legacy command/model
fallbacks; timeout covers provisioning; doctor follows the alias).
Explicit binaries no longer wait on the resolver lock, and callers
waiting for a shared download honor their own context.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…g by the timeout

Same compatibility rules as the Python and Go SDKs: implicit default +
AFORGE_BIN or aforgeBin keeps the aforge adapter, legacy command/model
fallbacks, the timeout covers first-use provisioning, and the doctor
follows the alias.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Without AFORGE_BIN the aforge provider runs CodeAF, so the doctor checks
CodeAF first and falls back to the legacy binary; with AFORGE_BIN it
checks that binary.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…fig; doctor honors CODEAF_BIN

A HarnessConfig whose provider was filled from AGENTFIELD_HARNESS_PROVIDER
at construction keeps that provider (as on main) instead of being treated
as the implicit default. The aforge doctor no longer falls back to the
legacy binary when an explicit CODEAF_BIN is missing, matching execution.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…commands, doctor honors CODEAF_BIN

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…s CODEAF_BIN

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The F3c tests compared two timestamps taken at different moments with
10-50 ms tolerances, which failed under load (seen locally with coverage
on). Assert the ordering instead: at least the provisioning time is
deducted from the budget, and the inner CodeAF timeout is the remaining
budget minus five seconds, taken no later than the outer one.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@AbirAbbas
AbirAbbas merged commit 647e31c into main Oct 24, 2026
50 checks passed
@AbirAbbas
AbirAbbas deleted the feat/codeaf-default-harness branch October 24, 2026 13:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants