Ephemeral previews: capture the worktree diff, guard the silent tool-loop exit, withhold web_search on confined runs - #1692
Merged
Conversation
…loop exit, withhold web_search on confined runs propose_code_change preview runs were coming back "completed" with the model's narration as the answer and no diff, while the edit had actually landed in the worktree. Root cause, confirmed from swarm38 logs: the Bifrost gateway rewrites the web_search_20250305 server tool to a newer version, the Anthropic API then implicitly offers code_execution, Sonnet answers with a server_tool_use the SDK never registered, `ai` marks the call invalid with a tool-error that lacks providerExecuted, the loop's client call/output counts disagree, and streamText exits with no error and no stop condition. get_context saw a last step with tool calls, skipped the stall nudge, fell through to extractFinalAnswer's all-text fallback, and reported success. Three changes, each sufficient on its own for the observed failure: - tools.ts: web_search is withheld on confined runs (create_pr worktrees and ephemeral previews). Those runs edit files and have no use for it, and with no server tool in the request there is nothing for a gateway to rewrite. - agent.ts / utils.ts: get_context detects a last step with unresolved tool calls (unresolvedToolCalls: invalid, unregistered, or without an output), retries once with those calls stripped from the transcript (stripToolCallParts — the API rejects a tool_result for a server tool it never ran) and a nudge naming the unavailable tool, and throws if the retry ends the same way. Two related holes closed on the same path: a mid-run abort now surfaces as AbortError instead of a completed run, and exhausted stall/length continuations set `incomplete` on the ContextResult so a caller that needs a deliverable can refuse it. - git_pr.ts / index.ts: captureWorktreeDiff stages the ephemeral worktree (git add -A, so new files are included — `git diff HEAD` alone omits them) and returns `git diff --cached`, with landChange's caps and gitleaks fail-closed scan. The non-stream terminal result carries it as `diff` / `diff_files` / `diff_error`, next to `pr`, so the caller no longer depends on the model pasting a diff into its final message. Tests: unresolved_tools.test.ts (helpers, including the exact production step shape), worktree_diff.test.ts (real git fixtures), and confinement tests asserting web_search is absent on ephemeral and prMode runs. The streaming path is untouched; Hive uses the non-stream path.
Same ok / failure / error shape as the create_pr result, one field instead of three, so the two run types read alike on the terminal result.
Evanfeenstra
added a commit
to stakwork/hive
that referenced
this pull request
Sep 15, 2026
… refuse incomplete runs (#5308) * propose_code_change: take the diff from the swarm's worktree capture, refuse incomplete runs Preview runs were coming back with narration instead of a diff while the edit had landed in the swarm's worktree (stakwork/stakgraph#1692 has the root cause: a gateway tool-version rewrite made the AI SDK's tool loop exit silently, and the swarm reported success). The swarm now reads the diff from the ephemeral worktree itself and returns it as `diff`, with `diff_error` when it looked and found a reason there is none, and `incomplete` when the run never reached a proper termination. - Prefer the swarm's `diff` over anything in the model's text. It is ground truth and includes new files; the model's pasted diff was a transcription that `git diff HEAD` would have left new files out of. - Refuse `incomplete` runs outright: the text is narration and the worktree may hold a half-applied change, so no proposal card. - Believe `diff_error` (no_changes, change_too_large, secrets_detected) over a diff the model pasted, with a specific message for each. - Keep the regex-on-text extraction as the fallback for swarms that predate the capture, and make that fallback stage first (`git add -A && git diff --cached`) so it too includes new files. - Surface the swarm's error text when the run fails instead of a generic "timed out or returned an error". Tests cover each source and refusal, the error passthrough, and the prompt change. Backward compatible with older swarms. * Read the swarm's preview diff from `preview`, mirroring `pr` The swarm reports the ephemeral worktree diff as one `preview` field with the create_pr result's ok / failure / error shape, instead of three top-level fields.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
propose_code_changepreview runs (Hive →/repo/agentwithephemeral: true) were returning completed with the model's narration as the answer and no diff, even though the edit had landed in the worktree. Confirmed fromrepo2graph.sphinxlogs on swarm38, 2026-09-15.Root cause chain:
web_search_20250305server tool to a newer version on the way to Anthropic (acknowledged by a Bifrost dev on Discord, 2026-07-07: "tool version is the difference").code_execution; Sonnet answered with aserver_tool_usefor it (text-editorstr_replace).@ai-sdk/anthropiconly tolerates that when it sentweb_search_20260209itself, soaimarked the call invalid and emitted atool-errorwithoutproviderExecuted.streamTextexited: no error, no stop condition, finish reasontool-calls.get_contextsaw a last step with tool calls, skipped the stall nudge, fell toextractFinalAnswer's all-text fallback, and reported success.Changes
tools.ts— withholdweb_searchon confined runs. create_pr worktrees and ephemeral previews edit files; they have no use for web search, and with no server tool in the request there is nothing for a gateway to rewrite.agent.ts/utils.ts— guard the silent exit.unresolvedToolCallsfinds calls on the last step that are invalid, unregistered, or without an output.get_contextretries once with those calls stripped from the transcript (stripToolCallParts; the API rejects atool_resultfor a server tool it never ran) plus a nudge naming the unavailable tool, and throws if the retry ends the same way. Also on this path: a mid-run abort now surfaces asAbortErrorinstead of a completed run, and exhausted continuations setincomplete: { reason }on theContextResult.git_pr.ts/index.ts— capture the diff from the worktree.captureWorktreeDiffrunsgit add -A(new files included;git diff HEADalone omits them) thengit diff --cached, withlandChange's caps and gitleaks fail-closed scan. The non-stream terminal result reports it aspreview, the counterpart ofpron create_pr runs, with the same shape:{ ok: true, diff, filesChanged }or{ ok: false, failure, error }wherefailureis one ofno_changes,change_too_large,secrets_detected,git_failed. Callers no longer depend on the model pasting a diff into its final message.Terminal result additions (ephemeral runs)
Testing
src/repo/**/*.test.ts: 583 passedunresolved_tools.test.ts(includes the exact production step shape),worktree_diff.test.ts(real git fixtures), and confinement tests assertingweb_searchis absent on ephemeral and prMode runstsc --noEmitclean on touched filesNotes
stream_context) is untouched; Hive uses the non-stream path.landChange.preview/incomplete; it is backward compatible with swarms that predate this.