docs: correct 14 more ADR records, five of which claimed capabilities the tree does not deliver (BACKLOG #1503-#1505) - #1006
Merged
Conversation
…s the tree does not deliver
A multi-agent over-claim hunt attacked 125 ADR build claims from the opposite
direction to the first audit: assume the record is overstating, and try to
prove the artifact is absent, mention-only, unwired, stubbed, or narrower than
the ADR decided. Nine over-claims were alleged; five survived three defenders
each. Three of the five are real capability gaps and are filed here.
#1503 ADR 0075 AC-1 asserts a batched mark_done that does not exist, and
cites a test name that does not exist
#1504 ADR 0133 D3 content-triggered alerts are unreachable from a Handler
#1505 ADR 0089 Phase A claims a read-atom row never built, and names a
set_segment the Message API lacks
ALL FIVE SURVIVORS WERE SCOPE_MISMATCH, NOT ABSENT CODE. In each case the
feature is real and wired; the record asserts a wider scope than the code
delivers. That class is invisible to a symbol grep, which is what the first
audit and my own solo review both ran.
EVERY CLAIM IN THESE THREE ITEMS WAS RE-VERIFIED HERE WITH A PAIRED CONTROL,
because they came from subagents and would otherwise be measured elsewhere and
asserted here:
_mark_done_batched 0 control _route_handoff_batched 4
the cited golden-SQL test name 0 control def test_ in that file 11
mark_done across both 0075 test files 0
content_match in pipeline/alerts.py 0 control in alert_sinks.py 4
content_match in __init__.py 0 control __all__ entries 92
read_field in lens.py 0 control set_field 20
def set_segment in message.py 0 control def add_segment 1
A FOURTH NUMBER, #1506, WAS ALLOCATED AND IS BURNED. It would have said ADR
0129 cites a parity test carrying zero Postgres or SQL Server references. All
three limbs failed on checking: the file names both backends in its docstring;
ADR 0129 never cites it for parity and states plainly that parity is a CI gate;
and tests/test_store_schema_hash.py already asserts the processed_files DDL in
all three _SCHEMA sets. ADR 0129's record is honest. The finding came from a
critic whose other points held, which is why it was checked rather than taken.
Severity is conditional throughout per CLAUDE.md section 0: zero deployments.
Nothing mis-executes; what is wrong is the record.
Open-count control per docs/LEDGER-GATE.md: items 488 to 491, open 299 to 302,
closed 189 unchanged.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…0003, 0041, 0154)
The first of two batches from a multi-agent audit that re-checked all 172 ADRs
against the tree. Each correction was drafted by one agent that re-verified the
finding independently, then rejected or passed by a second agent that checked
the old_string matches byte-for-byte and that the NEW text does not itself
over-claim. Only these three passed on the first attempt; thirteen more were
sent back and are not in this commit.
0003 the ADR's own Built line still read "Nothing yet" while every §1
connector and the §3 database source ship and register themselves.
The index row was already correct from the earlier pass; this fixes
the half a reader hits when they open the ADR instead of the index.
It also corrects a second stale clause nobody had flagged: the line
quoted an enum placeholder comment that no longer exists.
0041 Status read "Proposed" with D2/D3 "staged", and the two section
headings still said "(planned)". All three shipped in 0.2.9. The
index row was already accurate, so the ADR body was the stale half.
0154 the mirror case: the ADR body was correct and the INDEX cell was
stale, still saying the ADR authorises increment A only with the
sync-reply deferred. Both increments merged 2026-08-01.
Superseded wording is kept and dated rather than deleted, per the ADRs'
append-only rule. Scope cautions from the audit are carried into the text:
0003 says outright that only symbols and registry registration were verified,
not test coverage, and does not claim "fully built".
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
… shipped
The second batch from the multi-agent audit. These eleven were REJECTED on
their first draft and revised. That rejection rate is the finding: 13 of 16
first-attempt corrections failed their checker, and not on style -- on
substance. Corrections arrive with the authority of a fix, so a wrong one is
harder to catch than the wrong original. One rejection read "ONE SENTENCE IS
FALSE, AND IT IS THE POSITIVE CONTROL".
FIVE OF THESE ARE OVER-CLAIMS -- the record asserted MORE than the tree
delivers. Every one is a scope mismatch rather than absent code: the artifact
exists and is wired, it just does less than the ADR decided. That class is
invisible to a symbol grep, which is what the earlier passes ran.
0075 AC-1 asserts route_handoff / transform_handoff / mark_done are all
batched. mark_done is not, and never was -- no _mark_done_batched
exists and mark_done never reads the flag. AC-1 also cites a test node
id no file defines, which survived because adr_analyze.py checks only
that the FILE exists, not the test name. A cited-but-absent test reads
as verified evidence. The ADR's own open item is flipped to RESOLVED,
DEFERRED, which is what the shipped code and two by-design test
comments already say. Filed as BACKLOG #1503.
0133 D3 claims content-triggered alerts are built. content_match exists
only on NotifierAlertSink -- absent from the AlertSink protocol, from
LoggingAlertSink, and from the authoring surface, and a @handler
receives only msg. No Handler can reach it. D1 and D2 ARE fully wired
on all three backends and the correction says so. Filed as #1504.
0089 Phase A is claimed built and adopted; its read atom never shipped. The
recogniser is invoked only for a bare expression statement, so an
assignment cannot reach it, and set_segment does not exist on the
Message API. Measured by running the lens, not by grep. Filed as #1505.
0150 decision item 6 names three surfaces for the client field; it is
missing on one of them.
0002 the mirror case: the Built line said "Nothing in this ADR is built
yet" while WP-13a, WP-13b and WP-15 all ship and are self-labelled
with those exact work-package numbers.
The other six (0004, 0007, 0017, 0028, 0161, 0167) understate what shipped.
0007 is the sharpest: both halves of the record said unbuilt while the whole
connections.toml feature ships, including a write seam, and CLAUDE.md section
1 already describes it as shipped.
TWO ARE NOT IN THIS COMMIT. 0179 and 0176 were rejected a second time -- 0179
on a provenance paragraph, 0176 on a splice defect the dry run could not see.
Neither is applied rather than applied unverified.
Revisers were instructed that a checker is not automatically right, and four
(0075, 0089, 0007, 0161) recorded disputes with evidence instead of complying.
0075's is the one that mattered: the checker's own suggested replacement
sentence carried a fresh factual error, and following it literally would have
put a third wrong claim into the paragraph whose whole job is to be exact.
Superseded wording is kept and dated throughout, per the append-only rule.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-on to #1003 (merged). Same subject, different and much stronger instrument.
#1003 was a solo review. It corrected 21 ADR records by finding code — every check asked can I find the artifact, so a record claiming a build it never did had no way to fail. This PR is the result of running that missing direction, plus three more adversarial passes over the whole corpus.
Five records claimed capabilities the tree does not deliver
Every one is a scope mismatch, not absent code. The artifact exists and is wired; it does less than the ADR decided. No symbol grep can find this class, which is why #1003 and the first audit pass both missed all five.
Accepted (built), AC-3 "WHEN a Handler emits acontent_match"content_matchis onNotifierAlertSinkonly — absent from theAlertSinkprotocol, fromLoggingAlertSink, and from the authoring surface; a@handlerreceives onlymsg. No Handler can reach it. D1 and D2 are fully wired on all three backends, and the correction says so. → #1504route_handoff/transform_handoff/mark_doneall batched, citing a named test_mark_done_batched;mark_donenever reads the flag. The cited test node id does not exist — it survived becauseadr_analyze.pychecks that the file exists, not the test name. A cited-but-absent test reads as verified evidence. → #1503set_segmentdoes not exist on the Message API. Measured by running the lens, not by grep. → #1505clienton three named surfacesSix more (0004, 0007, 0017, 0028, 0161, 0167) understate what shipped. 0007 is the sharpest: both halves of the record said unbuilt while the entire
connections.tomlfeature ships including a write seam, andCLAUDE.md§1 already describes it as shipped.The rejection rate is the real finding
13 of 16 first-attempt corrections failed their checker. Not on style — on substance. Each correction was drafted by one agent that re-verified the finding independently, then checked by a second that confirmed the
old_stringmatches byte-for-byte and that the new text does not itself over-claim. One rejection read: "ONE SENTENCE IS FALSE, AND IT IS THE POSITIVE CONTROL."A correction arrives with the authority of a fix, so a wrong one is harder to catch than the wrong original. Had these been hand-applied from the audit findings — which is what I was about to do before adding the checker stage — most would have landed unverified.
Revisers were told a checker is not automatically right. Four (0075, 0089, 0007, 0161) disputed complaints with evidence rather than complying. 0075's mattered: the checker's own suggested replacement sentence carried a fresh factual error, and following it literally would have put a third wrong claim into the paragraph whose entire job is to be exact.
What is deliberately not here
#1506was allocated and is burned. It would have said ADR 0129 cites a parity test with zero Postgres/SQL Server references. All three limbs failed on checking: the file names both backends in its docstring, ADR 0129 never cites it for parity and says plainly that parity is a CI gate, andtests/test_store_schema_hash.pyalready asserts the DDL in all three_SCHEMAsets. It came from a critic whose other points held — which is why it was checked rather than taken.Method and its limits
Four workflows, 191 agents, ~22M subagent tokens. Every agent was required to report a positive control that fired alongside any null result, and told the ADR 0124 story (
wait_for_ackreturns zero; the real knob isno_ack; the feature is built) so a null reads as a wrong needle before it reads as an absence.The over-claim hunt's own critic is worth quoting against it: it inverted the one-sidedness rather than escaping it. Three of the five survivors were already written down somewhere in the repo before the hunt ran (0075's unchecked box,
BACKLOG.md:698,BACKLOG.md:270). Only 0089 and 0150 are independent discoveries. A cleaner instrument would find an over-claim in an ADR with no ledger row and no unchecked box.Also unresolved and worth someone's attention: 0013 carries two documents under one index row, so both audits graded one file and never opened the other.
Severity is conditional throughout per
CLAUDE.md§0 — zero deployments. Nothing mis-executes; what was wrong is the record.Checks
pre-commit, all three commitsbacklog_status_check.py --quietlink_check.py docscontrol_char_check.pyruff/mypy/pytestRebuilt onto current
mainrather than pushed onto the old branch: #1003 squash-merged, so its commits are not ancestors and the old branch would have re-proposed merged work.🤖 Generated with Claude Code