Add an Atomic SDK harness for the hosted Planner - #210
Merged
Merged
Conversation
lavaman131
force-pushed
the
atomic-harness
branch
from
September 28, 2026 22:42
4c685ef to
115df25
Compare
lavaman131
added this pull request to stack #220
September 30, 2026 08:36
MaggieAppleton
approved these changes
Sep 30, 2026
lavaman131
force-pushed
the
atomic-harness
branch
from
October 1, 2026 15:08
08da2e5 to
7544ed0
Compare
MaggieAppleton
force-pushed
the
atomic-harness
branch
from
October 2, 2026 16:51
572d3e2 to
6df7db0
Compare
HARNESS=atomic embeds Atomic 0.9.23 in the server process through its headless SDK (createAgentSession, ModelRuntime, DefaultResourceLoader, and in-memory session and settings managers). It does not spawn the atomic CLI, use RPC mode, or alias Pi's runtime. Each harness session owns one Atomic AgentSession in a private temporary directory. Every shipped Atomic package is off, the tool allowlist holds only the host tools and a result tool, and the loader discovers no extensions, skills, prompt templates, or context files. A per-turn hook replaces the whole system prompt, so the model never sees Atomic's coding-agent preamble. The adapter fails the turn before any model request if the live session reports an extra extension, tool, context file, skill, prompt template, or system prompt. It checks the tools offered to the model again before every model request. Host tools round-trip through submitToolResult and settle on abort. Structured output uses a terminating result tool enabled only from the turn's JSON response format, which captures the calling model's arguments without a second inference. An unrecognized provider/model fails the turn instead of letting Atomic pick another model, and MODEL is now required for HARNESS=atomic as it is for Pi. HARNESS_AUTH is required and is either auto or ai-gateway. auto copies the host's Atomic login into memory and is refused off loopback; ai-gateway accepts only vercel-ai-gateway models. Credentials are never written to disk. The Pi auth checks now share the same table with unchanged messages. Docs cover selection, trust, credentials, and Atomic's caveats. Assistant-workflow: ralph (run 0cfa5220-f488-43ef-8bb5-2f54cf5141f2) Assistant-model: Claude Opus 5.5 Assistant-duration: 30m converged, estimated 65m Assistant-verification: bun test passed: 1683 pass, 2 PostgreSQL skips, 0 fail, including the atomic contract suite and the unchanged copilot-sdk and pi suites Assistant-verification: bun run types passed: all workspaces and E2E Assistant-verification: bun run ci passed: dprint, oxlint with no errors, tokens Assistant-verification: bun run fix passed: only formatting changes, inspected Assistant-verification: mutation checks passed: turning builtins back on, enabling skill discovery, or offering the result tool on plain turns each fail the atomic suite Assistant-verification: agent-browser E2E passed: a real Chopin server with HARNESS=atomic and a local stub model ran an @chopin turn that called read_plan and streamed the reply into Chat; the model was offered exactly PLANNER_TOOL_NAMES and Chopin's Planner prompt Assistant-verification: qlty smells passed: no blocking findings; complexity warnings match the Copilot adapter's closure shape User-preference: Build the Atomic harness on Atomic's headless SDK, which separates runtime from hosts Co-authored-by: Alex Lavaee <lavaman131@github.com>
The ralph run's research stage mapped Atomic's SDK onto HarnessV1: event translation, host tool round trips, the tool boundary, auth modes, and caveats such as no cross-process resume and oversized tool result truncation. Keep it with the branch so review can continue on another machine. The run was stopped during its first review round. Both reviewers had no blocking findings; reviewer B noted one doc nit: self-hosting.md names only ATOMIC_CODING_AGENT_DIR, but Atomic also honors legacy PI_CODING_AGENT_DIR for the auto-mode auth.json path. The Docker build could not be verified on this machine because its Docker Desktop network fails TLS to registry.npmjs.org (npm.pkg.github.com and api.github.com still work), so bun install inside the image cannot fetch the new lockfile entries. Assistant-workflow: ralph (run 0cfa5220-f488-43ef-8bb5-2f54cf5141f2) Assistant-model: Claude Opus 5.5 Assistant-duration: 72m abandoned, estimated 90m Assistant-verification: docker build unavailable: container TLS to registry.npmjs.org fails with UNKNOWN_CERTIFICATE_VERIFICATION_ERROR on this Docker Desktop host User-preference: Stop the workflow and hand off when the local environment blocks verification; continue on another machine Co-authored-by: Alex Lavaee <lavaman131@github.com>
Atomic reads PI_CODING_AGENT_DIR when ATOMIC_CODING_AGENT_DIR is unset, so the auto-mode auth.json lookup lists both variables in precedence order. Assistant-model: Claude Opus 5.5 Assistant-workflow: inline Assistant-verification: source check passed: @bastani/atomic 0.9.23 dist/config.js getEnvValue prefers ATOMIC_CODING_AGENT_DIR, then PI_CODING_AGENT_DIR, and a set variable disables the ~/.pi fallback Assistant-verification: dprint ci passed: bun run ci Co-authored-by: Alex Lavaee <lavaman131@github.com>
The release keeps the same five builtin packages, so BUILTINS_OFF still covers every one and the Planner session stays isolated to Chopin's host tools. Assistant-model: Claude Opus 5.5 Assistant-workflow: inline Assistant-verification: bun test passed: 1688 pass, 2 PostgreSQL skips, 0 fail, including the atomic contract and isolation suites Assistant-verification: bun run types passed: all workspaces and E2E Assistant-verification: bun run ci passed: dprint, oxlint, tokens, type scale, Impeccable baseline Assistant-verification: source check passed: @bastani/atomic 0.9.24-alpha.1 dist/builtin lists intercom, mcp, subagents, web-access, workflows, matching BUILTINS_OFF Co-authored-by: Alex Lavaee <lavaman131@github.com>
Closing an Atomic session while its first turn was still setting up ran cleanup before the working directory existed; setup then created it, saw the session closed, and stopped without deleting it. Delete the directory on that closed path, in a finally so a failed dispose still removes it. Reported by Maggie Appleton in review. Assistant-model: Claude Opus 5.5 Assistant-workflow: inline Assistant-verification: fail-before/pass-after passed: the new lifecycle test left a chopin-atomic-planner directory behind without the fix and passes with it Assistant-verification: bun test passed: 1824 pass, 2 PostgreSQL skips, 0 fail Assistant-verification: bun run types passed: all workspaces Assistant-verification: bun run ci passed: dprint, oxlint, tokens, design contract, design record, Impeccable (no new findings) Co-authored-by: Alex Lavaee <lavaman131@github.com>
0.9.25 is the stable release that rolls up the 0.9.25 prereleases, including SDK workflow run control and the native MCP client. It ships the same five builtin packages, so BUILTINS_OFF still turns off each one for the isolated Planner. Assistant-model: Claude Opus 5.5 Assistant-workflow: inline Assistant-verification: bun test passed: 1824 pass, 2 PostgreSQL skips, 0 fail Assistant-verification: bun run types passed: all workspaces Assistant-verification: bun run ci passed: dprint, oxlint, tokens, design contract, design record, Impeccable (no new findings) Assistant-verification: source check passed: @bastani/atomic 0.9.25 dist/builtin lists intercom, mcp, subagents, web-access, workflows Co-authored-by: Alex Lavaee <lavaman131@github.com>
MaggieAppleton
force-pushed
the
atomic-harness
branch
from
October 2, 2026 16:59
6df7db0 to
a390374
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds
HARNESS=atomic, aHarnessV1adapter (apps/server/src/harness/atomic/adapter.ts) that embeds@bastani/atomic0.9.23 in the Chopin server process through its headless SDK:createAgentSession,ModelRuntime,DefaultResourceLoader, and in-memory session and settings managers. It does not spawn theatomicCLI, use RPC mode, or alias Pi. Thecopilot-sdkandpiharnesses are unchanged.07c5852adds the adapter, its unit tests (atomic/adapter.test.ts), the shared contract suite (harness/atomic.contract.test.ts),harnesses.ts/config.tswiring and tests, and docs (docs/hosted-agent.md,docs/self-hosting.md,docs/architecture.md,README.md).878147akeeps the research notes that map the SDK toHarnessV1and list the caveats.4c685efdocuments thatautoalso honors the legacyPI_CODING_AGENT_DIR. Atomic checksATOMIC_CODING_AGENT_DIRfirst, thenPI_CODING_AGENT_DIR, and a set variable disables the~/.pifallback.This branch was rebased onto
main(99204aa). The only conflict wasbun.lock. I regenerated it from main's lockfile, and it matches the branch's original lockfile except for main'simpeccableentries.Security boundary
MODELfails the turn before any model request. Atomic would otherwise silently pick another model.createStructuredOutputTool, which makes a second model call.Auth modes
HARNESS=atomicrequires an explicitHARNESS_AUTH:autocopies the host's Atomic login into memory ($ATOMIC_CODING_AGENT_DIR, then$PI_CODING_AGENT_DIR, otherwise~/.atomic/agentover~/.pi/agent), plus env keys and ambient cloud credentials. Startup refuses it unlessSERVER_HOSTis loopback-only.ai-gatewayreadsAI_GATEWAY_API_KEYand accepts onlyvercel-ai-gatewaymodels. It is the only mode allowed on a public bind.No credentials,
models.json, ormodels-store.jsonare written to disk.Documented caveats
@bastani/atomic.auto, an in-memory OAuth refresh can rotate the host CLI's refresh token.!commandAPI-key entries inauth.jsonare executed.maxAiCreditsdoes not apply.@bastani/atomicadds more than 250 MB of dependencies. The rebuiltchopin-atomicimage is 2.1 GB.typebox1.3.27 while Chopin pins 1.3.7. Host tool schemas cross as plain JSON Schema, so the two versions never meet.Verification on this machine
These checks ran on a new checkout after the rebase. Earlier results from the original machine are in the
07c5852commit trailers.The Playwright suite does not exercise
HARNESS=atomic; its agent servers useAGENT=offor the fake harness. I tested the Atomic harness separately in a real browser against a real model.Browser test of the Atomic harness with a live model
I started a server on loopback with
AGENT=on HARNESS=atomic HARNESS_AUTH=auto MODEL=github-copilot/gpt-6-luna, using the E2E preloads (e2e/github.tsande2e/harness/preload.ts). Those preloads fake only GitHub's REST and OAuth endpoints and the GitHub MCP URL. Model traffic went to GitHub Copilot through the host's Atomic login. With agent-browser I signed in through the server's real OAuth callback, addedocto-org/scorein the UI, created a document, typed a sentence, and sent two@chopinturns.read_planandlist_pull_requests, then answered with the document's sentence and the fixture PR title "Restring the harp section". The fake MCP log showslist_pull_requestswas called forocto-org/scorewith a bearer token,readonly=true, andtoolsets=pull_requests.read_planand thenedit_plantwice to append a three-item list and keep the existing sentence. The document and the transcript were unchanged after a page reload.The server's
[agent]diagnostic listed exactly Chopin's 16 host tools pluschopin_submit_atomic_result. It showed nobash, filesystem, fetch, or Intercom tool. Nothing in~/.atomic/agentwas modified during the run.Assistant-verification: docker build passed:
docker build --no-cache -t chopin-atomic .completed; bothbun install --frozen-lockfilestages fetched from registry.npmjs.org (build 995 packages, production 908 packages)Assistant-verification: docker runtime import passed:
docker run --rm chopin-atomic bun -e "await import('./apps/server/src/harness/harnesses.ts')"and a direct import ofatomic/adapter.tsboth loaded@bastani/atomicin the production imageAssistant-verification: bun run fix passed: no changes beyond the doc edit; one existing oxlint warning in the untouched
github-tools.test.tsAssistant-verification: bun test passed: 1688 pass, 2 PostgreSQL skips, 0 fail across 190 files;
apps/server/src/harnessalone 127 pass, covering atomic, copilot-sdk and piAssistant-verification: bun run types passed: all workspaces and E2E
Assistant-verification: bun run ci passed: dprint, oxlint (0 errors), tokens, type scale, Impeccable design check
Assistant-verification: bun run e2e passed: 236 Chromium tests after confirming ports 8788, 8789, 8791, 8792 and 8797 were free
Assistant-verification: agent-browser E2E passed: live github-copilot/gpt-6-luna turns through HARNESS=atomic HARNESS_AUTH=auto ran read_plan, list_pull_requests over the fake GitHub MCP (bearer, readonly, pull_requests toolset) and edit_plan; the edit and transcript persisted across reload; ~/.atomic/agent was not modified
Assistant-verification: atomic source check passed:
@bastani/atomic0.9.23dist/config.jsreadsATOMIC_CODING_AGENT_DIR, thenPI_CODING_AGENT_DIRAssistant-model: Claude Opus 5.5
Assistant-workflow: inline
Assistant-duration: 12m converged, estimated none given
User-preference: Build the Atomic harness on Atomic's headless SDK, which separates runtime from hosts
Co-authored-by: Alex Lavaee lavaman131@github.com
Merge preparation
Rebased onto main after #236 and #212. The database lifecycle test now allows 20 seconds for its three server starts, rather than Bun’s default five seconds. Its server-readiness checks and all assertions are unchanged. The previously passing Atomic run took 4.986 seconds; the rebased run hit the five-second limit. Fresh CI passed on the updated head: validation, browser integration, and container build.