Repository navigation
Make the git clone/fetch/checkout timeouts configurable and state the LFS behavior - #78
Merged
Merged
Conversation
The Python node's git subprocesses had hardcoded timeouts, and the tightest one sat on the step that does the most work: `git checkout -B pr-review FETCH_HEAD` materializes the whole working tree with a 30 second ceiling. A repo whose checkout is inherently large — a Unity project with tens of GB of LFS-tracked assets — cannot finish in 30s, so every review of it dies there with no way to move the wall. Each git operation now resolves its ceiling from the environment (per-operation variable, then the PR_AF_GIT_TIMEOUT_SECONDS umbrella, then the built-in default; an unparsable or non-positive value logs one line and falls back). The checkout default becomes 600s, matching the clone ceiling it shares work with, and the PR-head fetch joins it. Two things follow from raising those ceilings: - Repo resolution moves off the event loop. `_resolve_repo` ran inline inside the async `review` reasoner, so a multi-minute clone stopped the node answering /health and the control plane marked the agent inactive. - A timeout now actually stops the work. `subprocess.run(timeout=…)` kills only the direct git child, so a `git-lfs filter-process` grandchild kept the captured pipes open (a 30s ceiling measured 46.2s against a real LFS repo) and went on downloading after the reasoner had given up. Every git command runs in its own session and the whole process group is killed, with the index/shallow lock files the dead command left behind removed so the next review of that workspace is not blocked by a stale lock. git-lfs smudge is now skipped explicitly: pr-af exports GIT_LFS_SKIP_SMUDGE=1 to every git command unless the operator set that variable themselves, so LFS-tracked files land as pointer files regardless of what the image happens to ship. Fixes #76 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The Go node ported app.py's literals, including the 30 second checkout ceiling. It now reads the same environment variables with the same precedence and the same defaults as the Python node — a test in each suite pins the names and defaults independently, so the two cannot drift — and exports GIT_LFS_SKIP_SMUDGE=1 unless the operator set it. Timed-out git commands are killed as a process group (Setpgid plus a Cancel that signals the group, behind a unix build tag, with WaitDelay as the backstop) and the index/shallow lock files left behind are removed, so a filter helper cannot hold the pipes open past the ceiling or block the next review of that workspace. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The new variables are only useful if they reach the node. Both compose files listed their environment explicitly, so nothing set in the .env next to them was forwarded; they now pass the git timeout variables, GIT_LFS_SKIP_SMUDGE and the two budget variables through, and run with `init: true` so a killed git helper reparented to PID 1 is reaped. Both package manifests declare the same variables — including go/, which is the manifest `af install` actually uses — and deliberately give the per-operation timeouts no manifest default, since a materialized 600 would shadow PR_AF_GIT_TIMEOUT_SECONDS. The README now states the LFS behavior instead of leaving it to be inferred from the Dockerfile: the workspace is a shallow clone, pr-af sets GIT_LFS_SKIP_SMUDGE=1 for every git command, LFS-tracked files are checked out as pointer files (which is all a review needs, since it reads the diff and the source text), and the runtime image ships git without git-lfs anyway. Wanting real LFS content means installing git-lfs in a derived image, setting GIT_LFS_SKIP_SMUDGE=0, and raising PR_AF_GIT_CHECKOUT_TIMEOUT_SECONDS to cover the download. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The review path's git subprocess timeouts were hardcoded, and the tightest one sat on the step
that does the most work:
git checkout -B pr-review FETCH_HEAD, which materializes the workingtree, had a 30-second ceiling. On a repo whose checkout is inherently large — the reporter's is a
Unity project with tens of GB of LFS-tracked assets — every review dies there, with no
documented way to move the wall:
Configurable ceilings
Every git command on the review path now resolves its timeout from the environment, with the
precedence
per-operation variable→PR_AF_GIT_TIMEOUT_SECONDS→ built-in default. Anunparsable or non-positive value logs one line and falls back to the default rather than
disabling the ceiling.
PR_AF_GIT_CLONE_TIMEOUT_SECONDSgit clonePR_AF_GIT_FETCH_TIMEOUT_SECONDSPR_AF_GIT_CHECKOUT_TIMEOUT_SECONDSPR_AF_GIT_DIFF_TIMEOUT_SECONDSgit diffPR_AF_GIT_TIMEOUT_SECONDSThe checkout default moves to 600s to match the clone ceiling: with
--no-checkouton theclone, the checkout is the tree materialization, so it deserves the same order of ceiling. The
Python node and the Go node read the same names with the same precedence and defaults, and a
test in each suite pins that contract independently so the two cannot drift.
Git LFS, stated rather than inferred
pr-af now exports
GIT_LFS_SKIP_SMUDGE=1into the environment of every git command it runs,unless the operator has set that variable themselves (any value, including
0, is passedthrough). So LFS-tracked files land in the review workspace as pointer files and no LFS content
is downloaded during a review — which is enough, since the review reads the diff and the source
text. This was already true by accident (the runtime image installs
gitwith--no-install-recommends, and the Debiangitpackage neither depends on nor recommendsgit-lfs); it is now true by construction, independent of the base image, and documented in theREADME,
.env.exampleand the package manifest.An operator who does want real LFS content installs
git-lfsin a derived image, setsGIT_LFS_SKIP_SMUDGE=0, and raisesPR_AF_GIT_CHECKOUT_TIMEOUT_SECONDSto cover the download.Two things the investigation turned up
The node went unresponsive during repo resolution.
_resolve_repo— clone, fetch, checkout —ran inline inside the async
reviewreasoner, so a multi-minute clone blocked the event loop andthe agent stopped answering
/health; that is theAgent pr-af marked inactive after 4 consecutive failuresin the issue. It now runs throughasyncio.to_thread. Raising the checkoutceiling to 600s would have made that window six times longer, so it had to move.
A git timeout did not actually stop the work. Measured against a real LFS repo with smudge
enabled, a 30s ceiling took 46.2s to surface, and
git-lfs filter-processwas still running —still downloading — after the reasoner had already failed:
subprocess.run(timeout=…)kills onlythe direct
gitchild, and the grandchild keeps the captured pipes open. Git now runs in its ownprocess group and the whole group is killed when the ceiling is hit (Go:
Setpgid+Cancel,with
WaitDelayas the backstop), so the failure is prompt and nothing is left behind eatingbandwidth and disk in the container.
Validation
Repro first: against a workspace whose checkout goes through a slow smudge filter — the
mechanism git-lfs uses —
_checkout_pr_branchonmaindies at exactly 30.0s with thereporter's error string. Then, on this branch:
main)PR_AF_GIT_CHECKOUT_TIMEOUT_SECONDS=10Assets/Gizmos/NavMeshLink Icon.pngis a 130-byte pointer — no LFS content fetchedgit-lfsinstalled andGIT_LFS_SKIP_SMUDGE=0, ceiling 30sgit-lfsprocess left behindPR_AF_GIT_CHECKOUT_TIMEOUT_SECONDS=900Gates (the CI jobs, run locally):
ruff check src/ scripts/clean;pytest125 passed (the onefailure,
test_hitl_review_gate.py::test_create_request_fails_fast_when_hax_wedges, fails thesame way on
origin/mainunder Python 3.10 and passes on CI's 3.11);go build ./... && go vet ./... && go test ./...clean andgofmt -lempty. Thedocker-buildjob was not run locally —no Dockerfile changed.
An adversarial review pass of the first draft is what turned up the compose passthrough, the
af installmanifest, and the stale-lock retry failure; all three are fixed here.Fixes #76
🤖 Generated with Claude Code