Skip to content

records(oracle-pins): the parity pin is transcribed in two files and nothing checks they agree -- three checkers stay green over a disagreement #2829

Description

@localai-org-maint-bot

Row: -

Owed by the sync cycle. .agents/specs/upstream-pin-advance-e126687.md lists it
under ## Owed.

Pre-existing, and made live by the pin advance
(#2817,
#2827). Found by the fresh review
of that change and reproduced independently before filing.

The gap

The vLLM revision is written in TWO places:

  • .agents/upstream-sync.md, in its ```parity-pin block, which
    tools/bench/serve_low_common.py reads and every benchmark driver asserts
    against;
  • .agents/oracles/vllm.md, in its ```oracle-pin block, which
    scripts/check-oracle-pins.py validates.

Nothing checks that the two agree. check-oracle-pins.py says so in its own
docstring, deliberately: "It does not read the vLLM parity pin out of
upstream-sync.md and compare it to vllm.md. Two transcriptions of one value
is the drift this registry exists to stop; vllm.md says so in prose and points
there. Teaching the checker to sync them would BLESS the duplication, not remove
it." That reasoning is right about the FIX and it leaves the hole open in the
meantime.

Measured

With upstream-sync.md at the new pin and oracles/vllm.md rolled back to
5559679229bc961848b121ccdeaa8fa5d79bec98, in a scratch mutation of an otherwise
clean tree, each rc read directly and the tree restored afterwards:

$ python3 scripts/check-oracle-pins.py        RC=0   oracle-pins ok (14 oracles pinned)
$ python3 -m unittest tests.tools.test_oracle_pin  RC=0   Ran 25 tests, OK
$ python3 scripts/check-agent-record.py       RC=0   agent record OK

Three green checkers over a tree whose two pin surfaces name different
revisions. test_oracle_pin cannot catch it — it reads upstream-sync.md only —
and check-oracle-pins.py cannot, by design.

Why it matters now rather than before

Until 2026-09-03 the value had not moved since 2026-07-26, so a disagreement had
no opportunity to appear. A pin advance is exactly the edit that can leave one
surface behind, and the next one will be made by somebody who did not write this.
The surfaces agree in #2827 — verified by hand, twice, and by the mutation above
being the ONLY way to make them disagree.

What would close it

The checker's own docstring names the better fix: stop transcribing. Either
oracles/vllm.md drops its pin and pin_label and the registry resolves them
from the ```parity-pin block at read time — which is AGENTS.md §Records'
"derived at read time and not stored" shape and adds no shared file — or, if the
registry must keep a literal for offline use, one assertion compares them and
says which file is authoritative. Do not add a second stored copy and a
checker that syncs it
, which is the outcome the docstring warns against.

Nothing was executed beyond the three checkers above. No build, no GPU, no lease.

FOLLOWING_AGENTS_PROTOCOL

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions