Skip to content

feat: add command_code as a runtime - #154

Merged
AbirAbbas merged 15 commits into
Agent-Field:mainfrom
devparanjay:feat/add-commandcode
Oct 20, 2026
Merged

AbirAbbas merged 15 commits into
Agent-Field:mainfrom
devparanjay:feat/add-commandcode

Conversation

@devparanjay

Copy link
Copy Markdown
Contributor

Summary

  • What changed: Adds command_code as a runtime, backed by a new AgentField command-code harness provider that drives the Command Code CLI (cmd -p) headlessly. The provider is registered by wrapping the SDK's provider factory from swe_af/runtime/ — the same technique the existing codex patch uses — so upstream AgentField stays untouched. Also adds command_code to the runtime Literals in the build/execution/fast/issue configs, installs the CLI in the image, and mounts the host's ~/.commandcode into the agent containers.
  • Why it changed: To expand the available runtimes for SWE-AF. The goal was to make Command Code selectable alongside claude_code / open_code / codex. No model ids are baked in — a blank model omits -m so Command Code uses its own default, and any id it supports (now or later) passes through verbatim.

Validation

  • make check — 1311 passed, 1 skipped.
  • Relevant manual test performed — live headless run against the real CLI: NDJSON result parsed, .agentfield_output.json written and schema-validated through the AgentField runner, tokens/cache non-zero, and multiple full real build runs with no model config and expected results.

Behavior Impact

  • No behavior change
  • Backward compatible behavior change — purely additive; existing runtimes and their defaults are untouched.
  • Breaking change (explain below)

Notes

  • Auth: cmd login on the host plus the ~/.commandcode mount (mirrors ~/.codex). Hosts without bind mounts (Railway, etc.) need BYOK providers.json or a secret-mounted auth.json.
  • Follow-up: the Go node isn't covered — it still rejects command_code, since its SDK provider factory is a closed set.
  • tests/fast/test_integration.py::test_ac_14's allowlist was widened: the runtime set is declared in the config-schema modules, not in swe_af/runtime/ alone. Reasoner and coding-loop files
    remain disallowed.
  • Dockerfile installs command-code unpinned (latest).
  • Known limits: harness cost reports as unknown (token counts are recorded); per-role tool allowlists aren't expressible, so capability is governed by permission mode.
  • For a regular user who uses Command Code CLI and is already logged in, it just works - just set the default runtime to command_code. The user's configuration of Command Code on their machine is inherited by default, requiring minimal or no further configuration for SWE-AF or AgentField.

AC-14 asserts that `git diff HEAD` modifies no swe_af/ file outside
swe_af/fast/. That guard predates this work and fails for any runtime change,
because the runtime set is declared as a pydantic Literal in the config-schema
modules rather than in swe_af/runtime/ alone.

Permit swe_af/runtime/** and the two schema modules that declare the runtime
Literal. Reasoner and coding-loop files stay disallowed.
…provider

Register a new AgentField harness provider, "command-code", by wrapping the
SDK's provider factory from swe_af/runtime (the same technique
codex_harness_patch already uses), and add "command_code" to RUNTIME_VALUES
with its aliases and provider/adapter mappings.

CommandCodeProvider drives `cmd -p --output-format json` headlessly: models and
#variant reasoning effort, permission modes, max turns, session resume, cwd and
extra directories, system prompt inlining, plus mapping of the CLI's NDJSON
result line and documented exit codes onto RawResult.

No model ids are baked in - a blank model omits -m so Command Code uses its own
configured default, and any id it supports passes through verbatim. Token usage
is read from the CLI's camelCase fields, which the SDK's snake_case
extract_token_usage does not recognise.
Widen the runtime Literal on BuildConfig, ExecutionConfig, FastBuildConfig and
IssueBuildConfig, and give the runtime an all-empty base-model entry so model
resolution defers to Command Code's own default rather than pinning an id.
Caller-supplied models still win verbatim.
…ocument the runtime

Install command-code in the image, and mount the host's ~/.commandcode into the
agent containers the same way ~/.codex is mounted, so a host `cmd login` is the
only auth step. Cover it with a compose assertion and document the runtime in
the README, the deployment guide and .env.example.
@CLAassistant

CLAassistant commented Sep 26, 2026 •

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@devparanjay

devparanjay commented Sep 26, 2026 •

Copy link
Copy Markdown
Contributor Author

Just to add, I manually tested this PR branch by using it to make a simple web project - throwball.
It reached an impressive result in just the first run, but it took 2 more build runs to sort out minor issues and reach the final outcome.
In total, it consumed 269M+ tokens.

@ZeroPoint95 ZeroPoint95 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@devparanjay, thanks for adding the Command Code runtime. Two issues still need fixing:

  1. High — the Docker image cannot start the installed CLI. Dockerfile:23 installs current Command Code 1.82.2, which requires Node 22+, while the image’s Debian package supplies Node 20.19.2. I inspected the published entry point and reproduced its startup rejection using an isolated version-guard test. Please install a supported Node version and add a credential-free startup smoke test.

  2. Medium — configured executable paths are ignored. Provider construction:380 always creates CommandCodeProvider() without using _resolve_bin. SDK-runner probes confirm that SWE_COMMANDCODE_BIN and per-call binary overrides still launch cmd. On Windows, that also selects the system command shell. Please connect the resolver to execution, use a portable default, and test the overrides through the public runner.

make check passed: 1,311 tests passed and 1 skipped. These reproductions used mocked dispatch and the isolated startup guard; no Docker image build, real CLI execution or provider authentication was performed.

devparanjay and others added 6 commits October 18, 2026 14:06
Copy Node and npm from the official Node 22 image, install runtime libraries, pin Command Code to verified 1.82.2, and fail builds if its credential-free startup check fails.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Resolve per-call executable options and SWE_COMMANDCODE_BIN in the factory used by HarnessRunner, default to the non-colliding command-code npm alias, and cover dispatch, structured output, Windows stdin, and a real custom subprocess.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Include the SDK runner transient marker for exit code 6. Exercise retries for all transient exit codes through HarnessRunner and ensure auth, permissions, and credits do not retry.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Replace automatic writable home-directory mounts with opt-in read-only auth.json overlays. Keep CLI session storage writable inside containers, verify all mount definitions, and document credential visibility and the Python-only runtime.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The currently published Claude Code package declares Node >=22 for npm installation. Supply official Node 22 in the Go runtime image too and smoke-test the actual claude executable at build time; Command Code remains a Python-only runtime.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@AbirAbbas

Copy link
Copy Markdown
Collaborator

@devparanjay thanks for this, and sorry it sat. Since it had been open a while, I've taken it over and pushed the fixes on top of your branch (no history rewritten).

@ZeroPoint95, both of your points are addressed:

Node / Docker startup. The image now copies Node and npm from the official node:22-bookworm-slim image, pins command-code@1.82.2, and runs command-code --version during the build, so an incompatible Node fails the build instead of the first run. I installed that exact package in a scratch prefix: under Node 22 --version and --help work, and under Node 20.20.2 it exits 1 with "Command Code needs Node.js 22 or newer". go/Dockerfile doesn't install Command Code, but the Claude npm package it installs now also declares Node >=22, so that image gets the same Node 22 plus a claude --version check. (No Docker host on my side, so this is the real CLI run locally rather than an image build.)

Executable overrides. Provider construction now goes through the resolver: commandcode_bin > bin_path > SWE_COMMANDCODE_BIN > the package's command-code alias, so the default no longer collides with cmd on Windows. There are seven HarnessRunner.run cases covering that precedence, blank values and paths with spaces, plus Windows stdin and structured output. Against the previous head they all dispatch bare cmd and fail.

Two more things came up while verifying:

  • Exit code 6 (connection error) was classified as transient but never actually retried, because the SDK retries on the message text. It's fixed, with a test.
  • docker-compose.yml and docker-compose.local.yml were mounting the whole host ~/.commandcode read-write into every agent service, even when Command Code isn't the runtime. That directory holds auth, settings, MCP config, sessions and history, so a container could change host config. It's now opt-in through docker-compose.commandcode.yml / docker-compose.commandcode.local.yml, and it mounts only auth.json, read-only. The docs cover the setup.

make check on 3.12 passes (1,383 passed, 1 skipped), and the Go job's gofmt, build, vet and test -race are green. The one Go race failure I hit, cmd/furrowd TestChildServesRecordedStoreDirNotRawRunID, also reproduces on main (4 of 20 runs), and this PR doesn't touch Go code.

@AbirAbbas
AbirAbbas requested a review from ZeroPoint95 October 20, 2026 13:27
AbirAbbas
AbirAbbas previously approved these changes Oct 20, 2026

@AbirAbbas AbirAbbas left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified at c44db10: Command Code 1.82.2 starts under Node 22 and refuses Node 20 with the original error, executable overrides reach the public runner, exit 6 retries, and host credentials are now an opt-in read-only auth.json mount. make check (3.12) and the Go job are green.

AbirAbbas and others added 5 commits October 20, 2026 13:00
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Cover all five public build stages for each runtime, plus aliases and the legacy unknown-runtime fallback.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

@AbirAbbas AbirAbbas left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One more round before merging. A second, independent review turned up a few things, now fixed:

  • Fast builds ignored the runtime. swe-fast accepted command_code, but its own runtime-to-provider mapper only knew Claude and Codex, so every stage quietly ran on OpenCode. It now goes through the shared mapper, which keeps the old unknown-runtime fallback. A new test drives the real fast build and checks all five stages for each of the four runtimes; it fails on the previous head.
  • QA synthesizer coverage. The synthesizer's runtime table from #155 now has a command_code row.
  • Image compat. Both images add the nodejs → node alias that Debian's package provided. The changelog and deployment docs note that npm's global config moved from /etc/npmrc to /usr/local/etc/npmrc, with the override.
  • Docs. The runtime tables and lists include Command Code, marked Python-node only.

I merged current main in as well. make check (1,441 passed) and the Go job are green. Thanks again @devparanjay!

@AbirAbbas
AbirAbbas merged commit 5788dc7 into Agent-Field:main Oct 20, 2026
3 checks passed
@devparanjay

Copy link
Copy Markdown
Contributor Author

@ZeroPoint95 thanks for the review and apologies, has been difficult to find time to work on this recently.
@AbirAbbas No issues, things can be understandably busy at times. And thanks for taking it over, I probably wouldn't have been able to work on it this week either.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants