Skip to content

check-agent-record's issue-ownership gate exits 0 and prints nothing when its snapshot is absent, which is always the case in CI #2606

Description

@localai-org-maint-bot

Row: -

Owed by .agents/specs/unaligned-safetensors-consumers.md under ## Owed, added in #2602.

The gate is silent in CI, and only in CI

check_issue_index in scripts/check-agent-record.py reads the untracked snapshot scripts/agent-issue-index.py --refresh leaves behind. Its own docstring states the contract:

ABSENCE IS A SKIP, NOT A PASS. An absent or stale snapshot appends to skips with the command that fixes it and leaves errors alone. A gate that goes quiet when its input vanishes is #467 in a new place, so the caller must report the skip and --fail-on-skip must redden on it.

Neither half of that sentence is implemented. skips is populated and then never read: nothing prints it, nothing consults it before return 0, and --fail-on-skip does not exist -- check-agent-record.py --help offers only --report and --write-baseline.

.github/workflows/ci.yml:187 runs python3 scripts/check-agent-record.py --report, and no workflow step anywhere runs agent-issue-index.py --refresh:

$ grep -n "agent-issue-index" .github/workflows/*.yml
$ echo $?
1

So in CI the snapshot is always absent and the ownership check always does nothing, without saying so.

Measured, same tree, one file moved

At 872f7bfe4 (#2602), which cites an issue that at the time named no owning row:

$ python3 scripts/agent-issue-index.py --refresh && python3 scripts/check-agent-record.py
ERROR: .agents/issue-index.generated.md: this change references #2601, which names no
owning row. ...
rc 1

$ mv .agents/issue-index.generated.md /tmp/ && python3 scripts/check-agent-record.py --report
record anchors: ok=932, stale=28, broken=5  -> rot 33
agent record OK: ENGINE=178 MODEL=379 QUANT=86 KERNEL=58 BACKEND=88 ANCHOR-ROT=33
rc 0

Zero lines containing "skip" in the second run. The difference between a red gate and a green one is whether an untracked file happens to be on disk.

Why this matters more than the branch that found it

The polarity is backwards from every other gate here. A local operator who follows the documented procedure gets the red; CI, which is what decides whether work can land, gets the green. A change that violates the ownership rule lands green and the rule is enforced only against whoever happened to run --refresh first. That is the failure class this project names in #467, and it is the same class as the defect #2601 describes: an input that is absent reads as a result that is clean.

Fix shape

Two independent halves, and both are wanted:

  1. Implement the contract the docstring states. Print every entry in skips, and add the --fail-on-skip flag it names so a caller can make a skip red. A gate that cannot say "I did not run" is not a gate.
  2. Run python3 scripts/agent-issue-index.py --refresh in the CI step before check-agent-record.py, so the input exists where the verdict counts. The refresh is a network read against the tracker, so its failure mode wants deciding explicitly -- with (1) in place, a refresh that fails becomes a reported skip rather than a silent pass.

tests/scripts/test_agent_record.py should gain a red-before case for the absent-snapshot path, because the current suite passes with the check disabled.

Not in scope for #2602

Deliberately not fixed there. #2602 is a one-line test rename; a change to a record gate's semantics and to a CI workflow needs its own spec, a red-before test and a fresh review, and bundling it would hide it. Raised by the fresh review of #2602.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions