Skip to content

feat(mt#4866): Repair the selection and init/compile defects behind a zero-rule fresh project - #3623

Merged
edobry merged 7 commits into
mainfrom
task/mt-4866
Sep 4, 2026
Merged

edobry merged 7 commits into
mainfrom
task/mt-4866

Conversation

@minsky-ai

@minsky-ai minsky-ai Bot commented Sep 4, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Phase 0 of the accepted RFC "The rules Minsky ships" (Notion 3ce937f0, Accepted 2026-09-04 via
ask#11287). Five measured defects in the rules-selection and init/compile surfaces, plus a docs
correction. Every one is correct to fix under any outcome of the RFC's remaining open questions,
which is why this phase "activates now".

The defects compose into one user-visible failure: on a fresh Claude Code project, the first
rules disable a user ran wrote an unvalidated id into the committed config, and that config then
resolved to zero active rules — while a bare minsky compile wrote 6 .cursor/rules/*.mdc
files and a 90-byte AGENTS.md that nothing in the project reads, and any re-init deleted the
user's selection along with every other key init does not own.

All five defects were re-reproduced live at d667c9634 before being fixed, rather than
inherited from the 2026-09-01 measurement in the spec.

Key Changes

SC1 — rules enable|disable reject an unknown id (config-operations.ts). Validation lives in
the domain so CLI and MCP both get it, and runs before any config read/write so a rejection cannot
leave a partial rules: block. Valid ids are the on-disk .minsky/rules sources ∪
DEFAULT_TEMPLATES — the template half is deliberate, since a user may decline a rule init is
about to scaffold or one they deleted by hand, and neither appears in listRules.

SC2 — init --overwrite merges (init/config-merge.ts, wired in init.ts). Top-level merge:
keys init emits are refreshed, everything else is preserved. Deliberately not a deep merge —
mt#4699 stopped emitting tasks.strictIds and the mcp: block, and a deep merge would resurrect
them from old configs forever. createFileIfNotExists is untouched; its other callers
(setup.ts, mcp/registration.ts, every rule-file write) rely on plain overwrite semantics.

SC2 says "byte-for-byte"; here is exactly what the merge guarantees, measured. Every VALUE is
preserved exactly, and the result is byte-identical for input already in the serializer's canonical
block form — which is what init and rules enable|disable themselves write, so it holds for every
machine-generated config. It is not byte-identical for hand-authored input in other styles:
measured, disabled: [minsky-workflow] (flow style) round-trips to block style, and a comment line
is dropped. Both follow from the parse/stringify round-trip the criterion already scopes comment
preservation out of, and the criterion's substance holds in all cases — the rules: section and
every unowned key SURVIVE, where before they were deleted outright. A CST-level round-trip
(yaml's parseDocument) would buy byte-exactness and is deliberately not taken here: it would
preserve comments in init's merge while rules enable|disable strip them on the very next
command, which is a worse guarantee than a uniform one. Recorded on SC2 with mt#573 as the owner for
doing both together.

SC3 — cursor-rules-ts and agents.md are gated on the recorded harness (compile.ts, mirrored
in src/hooks/pre-commit.ts). Under workspace.harness: claude-code, a bare minsky compile no
longer writes them. Two escapes: an explicit --target never reaches the probe, and an output that
already exists stays maintained (per-target, not all-or-nothing). claude.md and claude-rules are
never gated — they are the two channels Claude Code implements (mt#3107). readRecordedHarness
fails open: an unreadable config gates nothing, so the failure direction is toward writing more.

SC4 — init's scaffolded set is pinned (init/rule-templates.ts). Six of seven registered
templates, now documented and pinned by a test that names the omitted one. Took the RFC's
"document the omission" branch — see ## Outcome on the task for why.

SC5 — removed an init --interface flag from the docs that has never existed. The 21 remaining
--interface occurrences belong to rules generate, where it is real and required.

SC6 — disabled subtracts from the corpus instead of emptying it (rule-selection.ts). The
active set now starts from the full corpus; presets/enabled add (intersected with the corpus),
disabled subtracts.

Consumer added by the READY gate, not in the original spec

src/hooks/pre-commit.ts's compileCheckTargets is a parallel implementation of
minskyCompileTargetsFromPresence — its own docblock says it is "kept in sync with" it. Gating only
the domain side would have made the pre-commit staleness check demand cursor-rules-ts /
agents.md outputs that a bare compile no longer produces, telling a claude-code project those
files are stale forever with no invocation able to refresh them. compile.test.ts and
compile-check-targets.test.ts were added to scope for the same reason. This is a no-op in this
repository
, which commits both outputs and so takes the already-exists escape.

Two decisions the criteria delegated, both recorded on the task

  • SC3 — agents.md is gated, not harness-agnostic. The RFC settles it: §Phase 0 says a bare
    compile "stops also writing .cursor/rules/ and AGENTS.md". Counter-argument (AGENTS.md is
    a cross-vendor convention) considered and rejected on evidence — the file init produced was 90
    bytes of generated header — and it loses cheaply, since creating the file once opts a project back
    in permanently.
  • SC6 — no allow-list mode is introduced. Consequence stated plainly because it is user-visible:
    at Phase 0 presets/enabled are inert, because the base set is already the whole corpus.
    That follows from the Phase-0 substitution (tier defaults do not exist until Phase 1a), not from
    an independent design choice. The corpus intersection is not inert even now — RULE_PRESETS
    names 13 Minsky-only ids and 3 that exist nowhere.

Scope condition recorded for Phase 2

The RFC says the active set "starts from the tier defaults at the project's rung". Those do not
exist at Phase 0, so "the full corpus" is the degenerate case of that sentence. mt#573 must
re-derive the base set from tier defaults rather than inherit "full corpus" as if the RFC said it
—
recorded on SC6, in the resolveActiveRules docblock, and in the task's ## Outcome.

Testing

Full main suite green, plus per-criterion live verification against real scratch projects.

Execution evidence:

$ bun scripts/run-tests-main.ts
 17571 pass
 0 fail
Ran 17581 tests across 1137 files. [326.88s]

$ bun scripts/run-tests-gated.ts
→ Change-scoped against merge base d667c9634.
 218 pass
 0 fail
Ran 218 tests across 15 files. [2.04s]
run-tests-gated.ts: all test steps passed.

$ bun scripts/run-related-tests.ts <all 7 changed source files>
 383 pass
 0 fail
Ran 383 tests across 26 files. [4.44s]

AT1 (rules disable --id no-such-rule exits non-zero, prints the id, config unchanged) — run
through the real CLI in a scratch repo, both directions:

=== md5 BEFORE ===  a0e04dece4c9440fc534bfaafa357a7f
=== (a) UNKNOWN id ===
Validation error: Unknown rule id "no-such-rule" — it is neither a rule in .minsky/rules nor a
rule minsky init can scaffold, so nothing would be selected. The project config was not written.
EXIT CODE: 1
=== md5 AFTER ===   a0e04dece4c9440fc534bfaafa357a7f      <-- unchanged
mentions id: 0
=== (b) KNOWN id (positive control) ===
✅ Success ... disabled: ["minsky-workflow"]
EXIT CODE: 0                                              <-- still writes

Pre-fix control, measured at d667c9634 on the unmodified tree: returned
{"enabled":[],"disabled":["no-such-rule-xyz"]} and persisted the id into .minsky/config.yaml.

AT2 (re-init preserves rules: and an unrelated key; tasks/persistence refreshed):

BEFORE: tasks.backend: minsky | rules.disabled: [minsky-workflow] | someUnrelatedKey: preserve-me
$ minsky init --repo <scratch> --backend github-issues ... --overwrite
AFTER:  tasks.backend: github-issues   <-- REFRESHED
        rules.disabled: [minsky-workflow]  <-- preserved
        someUnrelatedKey: preserve-me      <-- preserved

The refreshed tasks.backend is the discriminating half: a merge that preserved everything by
declining to write would also have kept the user keys. Pre-fix control at d667c9634: both
rules: and someUnrelatedKey were deleted.

Round-trip fidelity measured directly, both styles:

case A — canonical block input:  byte-identical output
case B — "rules:\n  disabled: [minsky-workflow]" plus a "# a user comment" line
         -> "rules:\n  disabled:\n    - minsky-workflow\n"   (flow normalized, comment dropped)

AT3 (claude-code scratch repo, no .cursor/, no AGENTS.md; bare minsky compile):

$ minsky compile
[compile] Target "claude.md": 1 file(s) written
[compile] Target "claude-rules": 0 file(s) written
AT3 PASS: .cursor/rules does not exist
AT3 PASS: AGENTS.md does not exist

$ minsky compile --target cursor-rules-ts     # escape (a)
-> .cursor/rules file count: 6

# escape (b): with .cursor/rules now present, a bare compile maintains it
$ minsky compile
[compile] Target "cursor-rules-ts": 6 file(s) written
-> AGENTS.md still absent            <-- gates are per-target, not all-or-nothing

Pre-fix control at d667c9634: the probe returned
["cursor-rules-ts","claude.md","agents.md","claude-rules"].

AT4 (a unit test asserts the scaffolded id set exactly) — rule-templates.test.ts, 5 pass. It
also pins which template is omitted, so adding a template to the registry without deciding whether
init scaffolds it fails here. Pre-fix control: no such test existed.

AT5 (grep finds no init --interface):

$ grep -n -- 'init --interface' docs/rules/template-system-guide.md
PASS: zero occurrences of 'init --interface'
$ grep -n -- '--interface' docs/rules/template-system-guide.md | wc -l
21      # all `rules generate`, where the flag is real and required

AT6 (rule-selection.test.ts gains a disabled-only case) — 10 pass. Run against the
unmodified resolver first, as AT6 requires; the failure is recorded below.

SC5 is the one mechanically-executable success criterion; its command and output are the AT5
block above.

Negative control — SC6 resolver: the new disabled-only case run against the unmodified resolver

error: expect(received).toEqual(expected)
- Set { "a", "b" }
+ Set {}
(fail) resolveActiveRules > mt#4866: a lone `disabled` entry subtracts from the full corpus...
 8 pass | 1 fail

Negative control — SC1 validation: assertKnownRuleId neutralized, restoring the full pre-fix
behaviour

 3 pass
 4 fail

The 4 failures are exactly the reject-asserting cases; the 3 that still pass are accept-cases, which
pass either way — correct, since accepting a valid id is the unfixed behaviour too.

For SC2 and SC3 the controls are the live pre-fix measurements against the genuinely unmodified
tree
at d667c9634, quoted under AT2 and AT3. Per mt#4512 that is stronger than a partial revert,
which risks leaving a state that is neither pre-fix nor post-fix.

Typecheck: 0 errors across 8 projects (infra/ skipped — deps not installed locally; CI covers it).
Lint: 0 errors, 0 warnings across 4382 files.

Live verification

The end-to-end runs above are the live verification: real minsky init / minsky compile /
minsky rules disable invocations against scratch projects under a sandboxed HOME, not
in-process fakes. No deployed service is exercised by this change.

Not structural under /implement-task §7a — no new persistence path, model-output channel,
external-system probe, deploy-target wiring, or schema migration — so no separate verification
artifact ships. The two pure functions (mergeProjectConfigYaml, resolveActiveRules) carry full
behavioural coverage; the two fs-touching paths carry the live runs above.

Deploy verification: packages/domain/** and src/** are deploy surface per
isDeploySurfaceFile() — verified by running the predicate over the changed files rather than
recalling the pattern list, which is why no commit here carries [no-deploy-impact]. The
post-merge deploy will be verified per §10 with notBefore set to the merge timestamp and
expectCommitSha set to the merge commit.

Testable-design note

enableRule/disableRule reach RuleService and fs directly rather than taking them injected.
Rather than patch those collaborators with spyOn, config-operations.test.ts gives them a real
scratch workspace
(mkdtemp + afterEach cleanup) — no module patching at all, which is the
better option under testing-standards.mdc §Testable Design. It carries a scoped
eslint-disable custom/no-real-fs-in-tests with that rationale, matching the existing precedent in
the sibling crud-operations.test.ts. Extracting an fs seam would change exported signatures with
live consumers, which RFC Phase 0 does not scope.

Sequencing

PR #3253 (mt#3854, IN-REVIEW) touches the same compile.ts / compile.test.ts /
pre-commit.ts / compile-check-targets.test.ts — verified at file level via get_files, not
inferred from its title. The two changes are opposite in direction (it adds two presence-gated
codex-* targets; this tightens two ungated ones), so this is a rebase rather than a semantic
conflict. SC1, SC2, SC4, SC5 and SC6 touch no file any of the 17 open PRs changes.

Full planning audit, premise checks and per-gate verdicts: mt#4866 ## Planning Audit (READY).

🤖 Generated with Claude Code

https://claude.ai/code/session_012aTeW8XXjVuEF5GykykSwq


Review round 1 — response

The reviewer's structured findings channel was empty; mt#2685 synthesized one
BLOCKING placeholder from the conclusion prose. The prose carried two real concerns,
so this was not treated as a malformed-review bypass case. Both are addressed.

R1-a — init's merge silently destroyed a config it could not read. Fixed; this was
the more serious of the two.
Two paths: an unreadable file warned and overwrote,
and an unparseable file returned the fresh content with no warning at all. My
own code comment defended the second as "recoverable, and matches the pre-mt#4866
behaviour" — but pre-mt#4866 behaviour is precisely the data loss SC2 exists to
stop, and it is not recoverable for the user's keys. It was worst on the path where
it is least visible: a --overwrite in CI, where the warning reaches nobody and the
command still reports success.

Both paths now throw UnmergeableConfigError naming the file and the remedy. SC2
requires every unowned key to survive; when the file cannot be read there is no way
to honour that, so refusing is the only faithful outcome. A file that parses but
holds no mapping still does not throw — there is nothing to lose, and refusing would
block a legitimate re-init over an empty config. That discriminating pair is tested.

Verified live on a config carrying user keys plus a deliberate syntax error:

BEFORE md5: 607f1b171afc8efdca26a1bb9848cfa3
$ minsky init --repo <scratch> ... --overwrite
Error: Cannot merge into the existing config at <path>: it is not valid YAML, so the
keys `minsky init` does not own cannot be preserved. Refusing to overwrite — that
would silently discard them.
Repair the file, or move it aside and re-run `minsky init --overwrite`.
EXIT: 1
AFTER  md5: 607f1b171afc8efdca26a1bb9848cfa3      <-- untouched

Before this round, that same input was replaced and reported success.

R1-b — the harness gate was invisible. Fixed. The stated mechanism ("when no
harness is recorded, behavior diverges") does not hold: with no harness both the
compile probe and the pre-commit check fail open identically, which is the
pre-mt#4866 set. But the underlying point is right — a gated-out target simply
vanished from the checked set, which reads identically to a target that was never
applicable, so a project could be told its outputs are current while two of them
were not being maintained at all.

Added minskyCompileTargetsWithGateReport / compileCheckTargetsWithGateReport.
The pre-commit check now prints which targets it is not checking and why:

ℹ️  Not checking cursor-rules-ts, agents.md — this project records
workspace.harness: claude-code and does not have those outputs on disk, so a bare
`minsky compile` does not produce them either (mt#4866). Run
`minsky compile --target <name>` to opt in.

The report-returning function is the implementation and the list-returning one
delegates to it, with a test pinning that the two agree, so they cannot fork.

Documentation impact — checked at source, not assumed. The finding named four
areas as likely falsified and stated "I did not read other compile-related docs to
verify consistency" and "I did not open those docs". I read each:

Claimed risk Verified
Re-init semantics Only docs/rules/template-system-guide.md mentions --overwrite, and it is rules generate --overwrite — a different command. Not falsified.
enable/disable validation Zero docs mention rules enable or rules disable. Nothing to update.
Resolver semantics architecture.md:299 and theory-of-operation.md:159 make location claims ("stored in .minsky/config.yaml under the rules key"), not semantics claims. No doc asserts allow-list behaviour. Not falsified.
Compile target gating No doc states which targets a bare minsky compile writes. Not falsified.

So nothing existing was made wrong. The underlying point — users need to understand
the new behaviour — stands regardless, so docs/rules/template-system-guide.md gains
three sections: re-init merge semantics including the refusal and the
formatting-normalization caveat, compile target gating with both escapes, and
enable/disable validation plus the Phase-0 inertness of presets/enabled.

Non-blocking items. Phase-0 resolver inertness: now documented in the guide, the
resolveActiveRules docblock, the PR body and the task's ## Outcome. Machine-readable
hint on fail-open harness read: covered by the gate report above. Shared gate helper
to avoid mapping drift: not taken, deliberately — importing the domain compile
module into a per-commit hook drags createMinskyCompileService and every target into
its import graph, which is the reason compileCheckTargets already mirrors rather than
imports. The duplication is documented on both copies; ADR-016 phase 4 (mt#2293)
consolidates the two pre-commit checks and is where it is properly resolved.
Known-id validation vs the future product corpus: validation reads DEFAULT_TEMPLATES,
so when Phase 1 replaces the scaffold set with the package corpus the valid-id set
follows it automatically.

Execution evidence (R1):

$ bun scripts/run-related-tests.ts <4 changed source files>
 309 pass
 0 fail
Ran 309 tests across 17 files. [3.02s]

Typecheck 0 errors across 8 projects; lint 0 errors, 0 warnings across 4382 files.

Negative control — R1-a fail-closed: the live run above IS the control in both
directions. The same input against the pre-R1 tree replaced the file and exited 0;
against this tree it exits 1 with the file byte-identical.

edobry and others added 6 commits September 4, 2026 13:42
…emptying it

`resolveActiveRules` started from the EMPTY set the moment any of presets/
enabled/disabled was non-empty. `disableRule` writes `disabled: [id]` with the
other two empty, so the first `rules disable` a user ran on a fresh project
resolved to ZERO rules. Reproduced live at d667c96 before the fix.

The active set now starts from the full corpus; presets and enabled ADD
(intersected with the corpus, which is not vacuous — RULE_PRESETS names 13
Minsky-only ids and 3 that exist nowhere), and disabled SUBTRACTS.

Phase-0 scope substitution is recorded in the docblock: the RFC says the base
set is "the tier defaults at the project's rung", which do not exist until
Phase 1a. mt#573 must re-derive it rather than inherit "full corpus".

Five existing cases asserted allow-list semantics and now assert additive ones;
their intent is unchanged, only the claim about non-members is inverted.

Deploy verification: packages/domain/** is deploy surface per
isDeploySurfaceFile(); verified post-merge per /implement-task §10.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012aTeW8XXjVuEF5GykykSwq
`rules enable|disable` pushed and filtered string arrays with no lookup against
any rule source, so `rules disable --id no-such-rule` returned success and wrote
the unknown id into the committed .minsky/config.yaml. With SC6's resolver that
config then resolved to zero active rules.

Validation lives in the domain (config-operations.ts), not the adapter, so the
CLI and MCP surfaces both get it, and runs BEFORE any config read or write so a
rejection cannot leave a partial `rules:` block behind.

Valid ids are the on-disk .minsky/rules sources UNION DEFAULT_TEMPLATES. The
template half is deliberate: a user may decline a rule init is about to scaffold,
or one they deleted by hand, and neither appears in listRules. Reading
DEFAULT_TEMPLATES rather than init's narrower hardcoded list keeps the seventh
template (minsky-session-management) selectable — see SC4.

Deploy verification: packages/domain/** is deploy surface per
isDeploySurfaceFile(); verified post-merge per /implement-task §10.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012aTeW8XXjVuEF5GykykSwq
…s flag that never existed

SC4. init scaffolded 6 of the 7 registered templates with no comment and no test,
so the omission was indistinguishable from an oversight. Took the RFC's
"document the omission" branch rather than scaffolding the seventh: every
scaffolded template is currently unreachable under Claude Code (mt#4735) and
three of the six carry confirmed-wrong instructions that mt#1230 owns, so a
seventh un-audited template writes more wrong prose into real projects — and RFC
Phase 1 retires the whole set anyway. minsky-session-management stays SELECTABLE
because SC1 validates against DEFAULT_TEMPLATES, not this list.

The list is hoisted to an exported INIT_SCAFFOLDED_RULE_IDS and pinned by a test
that also asserts exactly one registered template is omitted and names it, so
adding a template without deciding whether init scaffolds it now fails a test.

SC5. docs/rules/template-system-guide.md showed `minsky init --interface=cli|mcp`.
That flag has never been an init parameter; init derives the mode from --mcp.
Replaced with the real invocation plus a retraction. The 21 remaining
--interface occurrences all belong to `rules generate`, where it is a real and
required flag — unchanged.

Deploy verification: packages/domain/** is deploy surface per
isDeploySurfaceFile(); verified post-merge per /implement-task §10.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012aTeW8XXjVuEF5GykykSwq
… it does not own

init returns early when .minsky/config.yaml exists and --overwrite is absent, so
--overwrite is the ONLY re-init path, and it wrote the generated file
unconditionally. Measured live: a `rules:` block AND an unrelated
`someUnrelatedKey` both vanished. The loss is general, not rules-specific, so the
merge is by top-level key rather than a rules: special case.

Merge is TOP-LEVEL ONLY and the fresh value wins for any key it defines.
Deliberately not a deep merge: mt#4699 STOPPED emitting tasks.strictIds and the
mcp: block, and a deep merge would resurrect them from old configs forever.
Conditionally-emitted keys (repository, project) are preserved when this run does
not produce them, so a re-init without a git remote does not drop them.

Implemented in init, not in getMinskyConfigContentYaml (SC2 is explicit), and
createFileIfNotExists is untouched — setup.ts, mcp/registration.ts and every
rule-file write depend on its plain overwrite semantics.

Known limitation, recorded: the merge round-trips through the YAML parser, so
comments in the existing file are lost. SC2 scopes comment preservation out; the
same limitation already applies to rules enable|disable.

Deploy verification: packages/domain/** is deploy surface per
isDeploySurfaceFile(); verified post-merge per /implement-task §10.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012aTeW8XXjVuEF5GykykSwq
…ed harness

cursor-rules-ts and agents.md were selected on the presence of .minsky/rules
alone, so a bare `minsky compile` after a Claude-Code-only init wrote 6
.cursor/rules/*.mdc files and a 90-byte AGENTS.md that nothing in that project
reads. Measured live: with workspace.harness: claude-code recorded and neither
output on disk, the probe returned all four rules-sourced targets.

The RFC settles the agents.md half explicitly (Phase 0: "stops also writing
.cursor/rules/ and AGENTS.md"), so both are gated rather than leaving agents.md
harness-agnostic. claude.md and claude-rules are NOT gated — they are the two
channels Claude Code implements (mt#3107).

Two escapes, both required by SC3 and both verified live: an explicit --target
never reaches the probe, and an output that already exists stays maintained. The
second is per-target, not all-or-nothing, and is what makes this a no-op in
Minsky's own repo, which commits both outputs.

compileCheckTargets in src/hooks/pre-commit.ts gets the same gate. Its docblock
already required it stay in sync; leaving it would make the pre-commit check
demand outputs the compile no longer produces, telling a claude-code project its
.cursor/rules and AGENTS.md are stale forever with no way to refresh them. This
consumer was missing from the spec's scope and was added by the READY gate's
contract-propagation sweep.

readRecordedHarness fails OPEN: an unreadable config yields undefined, meaning
"gate nothing". The failure direction is toward writing more, not less.

Deploy verification: packages/domain/** and src/** are deploy surface per
isDeploySurfaceFile(); verified post-merge per /implement-task §10.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012aTeW8XXjVuEF5GykykSwq
@minsky-ai minsky-ai Bot added the authorship/co-authored Co-authored by human and AI agent label Sep 4, 2026
@minsky-reviewer

minsky-reviewer Bot commented Sep 4, 2026 •

Copy link
Copy Markdown

Minsky Reviewer Status

Verdict: APPROVED — no blocking findings
Review: View review
Model: openai/gpt-5 | Tokens: 400K prompt, 8K completion | Duration: 153s
Mode: normal

Commands

  • /review — request a fresh review

@minsky-reviewer minsky-reviewer Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Independent adversarial review (Chinese-wall)
Reviewer: minsky-reviewer[bot] via openai:gpt-5
Tier: 2


Request changes. While the PR addresses SC1–SC6 with coherent code and tests, two risks should be resolved before merge: (1) Pre-commit target gating now depends on recorded harness; when no harness is recorded, behavior diverges from claude-code expectations. Add an explicit designed observable/warning in the hook to flag missing harness so teams aren’t told to refresh outputs that a bare compile won’t produce. (2) Init’s merge fallback on unreadable config proceeds to overwrite with only a log warning; introduce a stronger, structured signal or fail-with-flag to prevent silent data loss in non-interactive/CI contexts. Non-blocking: clarify known-id validation vs future product corpus; consider a shared gate helper to avoid mapping drift between compile and hook; document Phase-0 resolver inertness for enabled/presets; add machine-readable hint when compile fails open on harness read. Docs likely need additional updates for compile target gating, overwrite-merge semantics, and enable/disable validation. Overall quality is strong; address the two blocking UX/data-safety concerns and this should be ready.

Findings

  • [BLOCKING] (review summary):1 — Reviewer concluded REQUEST_CHANGES but emitted no structured findings
    Synthesized by the empty-findings coherence recovery pass (mt#2685): the reviewer model called conclude_review with event=REQUEST_CHANGES but zero submit_finding calls, so the structured findings channel was empty even though the conclusion summary describes blocking issue(s) in prose. Original conclusion summary:

Request changes. While the PR addresses SC1–SC6 with coherent code and tests, two risks should be resolved before merge: (1) Pre-commit target gating now depends on recorded harness; when no harness is recorded, behavior diverges from claude-code expectations. Add an explicit designed observable/warning in the hook to flag missing harness so teams aren’t told to refresh outputs that a bare compile won’t produce. (2) Init’s merge fallback on unreadable config proceeds to overwrite with only a log warning; introduce a stronger, structured signal or fail-with-flag to prevent silent data loss in non-interactive/CI contexts. Non-blocking: clarify known-id validation vs future product corpus; consider a shared gate helper to avoid mapping drift between compile and hook; document Phase-0 resolver inertness for enabled/presets; add machine-readable hint when compile fails open on harness read. Docs likely need additional updates for compile target gating, overwrite-merge semantics, and enable/disable validation. Overall quality is strong; address the two blocking UX/data-safety concerns and this should be ready.

Spec verification

Criterion Status Evidence
SC1 — rules enable --id X / rules disable --id X reject an X that is neither a .minsky/rules/*.mdc source nor a shipped template id, with a non-zero exit and an error naming X; the config file is not written on rejection. Met packages/domain/src/rules/operations/config-operations.ts:105-151 adds assertKnownRuleId using on-disk RuleService ids ∪ DEFAULT_TEMPLATES, called at enableRule/disableRule entry. packages/domain/src/rules/operations/config-operations.test.ts covers rejection naming the id and ensures config unchanged.
SC2 — init --overwrite preserves an existing rules: section of .minsky/config.yaml byte-for-byte (amended: values preserved; canonical block form byte-identical) and preserves every top-level key init does not emit. Keys init emits are refreshed. Merge implemented in init, not by teaching getMinskyConfigContentYaml about rules:. Met packages/domain/src/init.ts:165-209 integrates merge path using mergeProjectConfigYaml and logs preserved keys; packages/domain/src/init/config-merge.ts implements top-level merge; packages/domain/src/init/config-merge.test.ts asserts preservation of rules: and unrelated key, refresh of emitted keys, non-deep-merge, and idempotence.
SC3 — In a project whose recorded harness is claude-code, a bare minsky compile writes no .cursor/rules/ and does not create AGENTS.md unless (a) --target is named, or (b) the output already exists. claude.md and claude-rules remain ungated. Probe must read recorded harness (fail-open). Met packages/domain/src/compile/compile.ts:38-84 gates cursor-rules-ts and agents.md when harness==='claude-code' unless existingOutputs true; readRecordedHarness reads workspace.harness from config (fail-open). packages/domain/src/compile/compile.ts:115-136 probeMinskyCompileTargets threads harness and existingOutputs. Tests in packages/domain/src/compile/compile.test.ts:133-259,259-358 assert gating, escapes, and fail-open behavior. Mirror added in src/hooks/pre-commit.ts:2403-2414,2583-2617 with tests in src/hooks/compile-check-targets.test.ts:108-170.
SC4 — init's scaffolded template set is either all seven, or six with a code comment naming why the seventh is excluded; a test pins the chosen set. Met packages/domain/src/init/rule-templates.ts:6-40 documents omission and exports INIT_SCAFFOLDED_RULE_IDS; generateRules uses it. packages/domain/src/init/rule-templates.test.ts pins exact six-order list and asserts exactly one omitted id equals minsky-session-management and remains selectable.
SC5 — docs/rules/template-system-guide.md:171-177 no longer documents an init --interface flag that does not exist. Met docs/rules/template-system-guide.md:172-188 replaces init --interface examples with --mcp true/false and adds an explicit note that init has no interface flag, citing src/adapters/shared/commands/init.ts parameters.
SC6 — resolveActiveRules treats disabled as subtractive over the full corpus when neither presets nor enabled is set; with presets/enabled set, they ADD to the base (intersection with corpus), and unknown ids never count toward active. Phase-0 degenerate base acknowledged. Met packages/domain/src/rules/rule-selection.ts:12-58 rewrites resolver to start from full corpus, add presets/enabled with intersection, and subtract disabled. Tests in packages/domain/src/rules/rule-selection.test.ts:24-113 update additive assertions and add disabled-only subtraction and unknown-id-not-added cases.

Documentation impact

  • blocking-needs-update — This PR changes documented behavior in multiple places beyond the single in-PR doc fix. While docs/rules/template-system-guide.md was corrected for the nonexistent init --interface flag, other user-facing behaviors changed and likely invalidate or require additions in their respective docs: (1) Compile behavior for claude-code projects: a bare minsky compile now gates cursor-rules-ts and AGENTS.md unless the outputs already exist; users must understand escapes and how to opt-in. I did not read other compile-related docs to verify consistency. (2) Re-init semantics: init --overwrite now merges rather than replaces, with value-preservation but possible style/comment normalization; any docs implying full overwrite are now wrong. I did not survey init guides beyond the edited page. (3) Rules enable/disable: unknown ids now reject with non-zero exit; any docs/tutorials showing permissive behavior or not stating validation should be updated. (4) Resolver semantics: presets/enabled now add to a full-corpus base and disabled subtracts; prior language implying allow-list semantics or that enabled narrows selection would be false at Phase 0. I did not open those docs; this is a correctness risk if current prose asserts prior behavior. Please update the relevant docs accordingly.
    Affected: docs/rules/template-system-guide.md

…face the harness gate

The reviewer's structured findings channel was empty (mt#2685 synthesized one
BLOCKING placeholder), but its conclusion prose carried two real concerns. Both
addressed; the doc-impact finding checked rather than assumed.

**R1-a (real, and the more serious). Init's merge silently destroyed a config it
could not read.** Two paths: an unreadable file warned and overwrote, and an
UNPARSEABLE file returned the fresh content with no warning at all. My own comment
defended the second as "recoverable, and matches the pre-mt#4866 behaviour" — but
pre-mt#4866 behaviour is exactly the data loss SC2 exists to stop, and it is
unrecoverable for the user's keys. Worst on the path where it is least visible: a
--overwrite in CI, where the warning goes nowhere and the command reports success.

Both paths now throw UnmergeableConfigError naming the file and the remedy. SC2
requires every unowned key to survive; when the file cannot be read there is no way
to honour that, so refusing is the only faithful outcome. A file that PARSES but
holds no mapping still does not throw — nothing to lose there, and refusing would
block a legitimate re-init over an empty config. Verified live: exit 1 with the
config md5 identical before and after.

**R1-b. The harness gate was invisible.** A gated-out target simply vanished from
the set, which reads identically to one that was never applicable — so a project
could be told its outputs are current while two were not being maintained at all.
Added minskyCompileTargetsWithGateReport / compileCheckTargetsWithGateReport; the
pre-commit check now prints which targets it is not checking and why. The
report-returning function is the implementation and the list-returning one
delegates to it, with a test pinning that they agree, so they cannot fork.

**Doc impact — checked, not assumed.** The reviewer flagged four docs as likely
falsified and said it had not read them. I read each: `--overwrite` in the template
guide is `rules generate --overwrite` (a different command); no doc mentions
`rules enable|disable`; architecture.md's and theory-of-operation.md's `presets`
lines are location claims, not semantics; no doc asserts allow-list resolver
behaviour. Nothing was falsified. Documented the three NEW behaviours anyway —
re-init merge semantics and the refusal, compile target gating with both escapes,
and enable/disable validation plus the Phase-0 inertness of presets/enabled.

Deploy verification: packages/domain/** and src/** are deploy surface per
isDeploySurfaceFile(); verified post-merge per /implement-task §10.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012aTeW8XXjVuEF5GykykSwq

@minsky-reviewer minsky-reviewer Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Independent adversarial review (Chinese-wall)
Reviewer: minsky-reviewer[bot] via openai:gpt-5
Tier: 2


R2 verification complete. The fix commit addresses the prior round’s concerns: (1) init’s overwrite path now fails closed on an unreadable/unparseable config via UnmergeableConfigError and performs a top-level merge that preserves unowned keys; comprehensive tests cover preservation, replacement, idempotence, and refusal. (2) The harness gate is surfaced end-to-end: compile target selection is harness-aware with explicit escapes and a parallel pre-commit implementation logs gated-out targets. Docs were updated to correct the init interface flag and to document SC2/SC3 behavior. I found no new critical defects introduced by this commit. Ready to merge.

Spec verification

Criterion Status Evidence
SC1 — rules enable --id X / rules disable --id X reject an X that is neither a .minsky/rules/*.mdc source nor a shipped template id, with a non-zero exit and an error naming X; the config file is not written on rejection. Met packages/domain/src/rules/operations/config-operations.ts:66-103 — assertKnownRuleId validates against on-disk rules plus DEFAULT_TEMPLATES, throws ValidationError naming the id before any write. enableRule/disableRule (lines 109-153) call it first, ensuring no partial writes on rejection.
SC2 — init --overwrite preserves an existing rules: section of .minsky/config.yaml byte-for-byte, and more generally preserves every top-level key init does not itself emit. Keys init does emit are refreshed. The merge is implemented in init, not by teaching getMinskyConfigContentYaml about rules:. (Byte-for-byte amended note applies.) Met packages/domain/src/init.ts:338-371 wires a merge path for --overwrite via mergeProjectConfigYaml, refusing to proceed when unreadable; packages/domain/src/init/config-merge.ts implements top-level, non-deep merge preserving unowned keys and refreshing owned ones; tests in packages/domain/src/init/config-merge.test.ts assert preservation of rules: and unrelated keys, whole-section replacement, idempotence, and fail-closed behavior on unparseable YAML.
SC3 — In a project whose recorded harness is claude-code, a bare minsky compile writes no .cursor/rules/ and does not create AGENTS.md unless (a) --target, or (b) output already exists. claude.md and claude-rules are never gated. Probe reads recorded harness; failure direction is open. Met packages/domain/src/compile/compile.ts:63-116 introduces harness-aware selection via minskyCompileTargetsWithGateReport, dropping cursor-rules-ts and agents.md under claude-code unless present already; packages/domain/src/compile/compile.ts:204-220 probeMinskyCompileTargets reads harness/output presence (with readRecordedHarness) and passes them; tests in packages/domain/src/compile/compile.test.ts:134-306 cover mapping and probe, including escapes and fail-open on unparseable config. Pre-commit mirror updated in src/hooks/pre-commit.ts:2392-2429 to apply the same gate and message.
SC4 — init's scaffolded template set is either all seven, or six with a code comment at the list naming why the seventh is excluded; a test pins the chosen set so a change is a deliberate edit. Met packages/domain/src/init/rule-templates.ts (earlier commit in this PR) documents the omission and derives interface; tests pin the scaffolded set (referenced in PR body as rule-templates.test.ts, which is green in CI). No regressions introduced by current fix commit.
SC5 — docs/rules/template-system-guide.md:171-177 no longer documents an init --interface flag that does not exist. Met docs/rules/template-system-guide.md:172-214 — the incorrect init --interface examples are removed/replaced with --mcp true|false and an explicit paragraph states that init has no interface flag, citing the command's real params.
SC6 — resolveActiveRules treats disabled as subtractive over the full corpus when neither presets nor enabled is set; presets/enabled add; unknown ids intersected with corpus. Scope substitution recorded. Met packages/domain/src/rules/rule-selection.ts (implemented in prior commit of this PR) starts from full corpus then subtracts disabled and adds presets/enabled intersected with allRuleIds. Tests updated per PR body (rule-selection.test.ts new disabled-only case); no regressions in current fix commit.

Adoption sweep

Symbol Kind Consumers found Classification Notes
packages/domain/src/compile/compile.ts — minskyCompileTargetsWithGateReport function packages/domain/src/compile/compile.test.ts:17 — imported and exercised in harness-gate tests, packages/domain/src/compile/compile.ts:100 — delegated to by minskyCompileTargetsFromPresence Adopted New API returns targets plus gated-out reasons; used by tests and the delegating legacy function keeps prior callers working.

Documentation impact

  • updated-in-pr — This PR updates docs/rules/template-system-guide.md to remove the incorrect init --interface examples and adds sections documenting SC2’s merge semantics and SC3’s harness-gated compile behavior. The code changes align with these updates (packages/domain/src/init.ts merge path; packages/domain/src/compile/compile.ts harness gate), so docs were revised in-PR to match the new behavior.

@edobry
edobry merged commit 382d255 into main Sep 4, 2026
20 checks passed
@edobry
edobry deleted the task/mt-4866 branch September 4, 2026 18:37

@minsky-reviewer minsky-reviewer Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Independent adversarial review (Chinese-wall)
Reviewer: minsky-reviewer[bot] via openai:gpt-5
Tier: 2


R3 verification complete. The prior BLOCKING concerns from R1 were addressed in the last iteration, and this round’s diff introduces the harness gating/reporting and init-merge wiring with comprehensive tests and docs. I re-checked SC1–SC6 against the current spec: id validation now guards writes and names the id; init --overwrite performs a top-level merge and fails closed on unreadable configs; compile target gating is harness-aware with per-target escapes and a mirrored pre-commit implementation that surfaces gated-out targets; the scaffolded set is pinned with rationale; docs remove the nonexistent init --interface flag and document the new behaviors; the resolver now starts from the corpus and subtracts disabled with corpus intersection. I found no new critical defects introduced by these changes. Ready to merge.

Spec verification

Criterion Status Evidence
SC1 — rules enable --id X / rules disable --id X reject an X that is neither a .minsky/rules/*.mdc source nor a shipped template id, with a non-zero exit and an error naming X; the config file is not written on rejection. Met packages/domain/src/rules/operations/config-operations.ts:78-140 adds knownRuleIds (on-disk ∪ DEFAULT_TEMPLATES) and assertKnownRuleId(); enableRule/disableRule now await assertKnownRuleId(...) before any read/write (lines 141-170). Tests in packages/domain/src/rules/operations/config-operations.test.ts cover rejection, message naming the id, and “config file not written” behavior.
SC2 — init --overwrite preserves an existing rules: section byte-for-byte (amended: values preserved exactly; serializer-canonical input byte-identical), and preserves every top-level key init does not itself emit; owned keys refreshed; implemented in init not the generator; refusal on unreadable config is acceptable (amended). Met packages/domain/src/init.ts:338-372 wires merge on overwrite via mergeProjectConfigYaml and throws UnmergeableConfigError on unreadable config. New module packages/domain/src/init/config-merge.ts implements a top-level merge preserving unowned keys, not deep. Tests in packages/domain/src/init/config-merge.test.ts assert preservation of rules: and unrelated keys, replacement of owned sections, idempotence, refusal on unparseable YAML, and preservedKeys reporting.
SC3 — In a project whose recorded harness is claude-code, a bare minsky compile writes no .cursor/rules/ and does not create AGENTS.md unless (a) target named with --target, or (b) output already exists; claude.md and claude-rules are never gated. Pre-commit check mirrors this selection and surfaces gated-out targets. Met packages/domain/src/compile/compile.ts introduces harness-aware gating via minskyCompileTargetsWithGateReport and readRecordedHarness; probeMinskyCompileTargets passes harness and existingOutputs. Tests in packages/domain/src/compile/compile.test.ts cover gating, escapes, and probe reading. src/hooks/pre-commit.ts switches to compileCheckTargetsWithGateReport, adds readRecordedHarnessForCheck, mirrors the same gating, and logs gated-out targets (lines ~2392-2427, 2604-2700).
SC4 — init's scaffolded template set is either all seven, or six with a code comment naming why the seventh is excluded; a test pins the set. Met packages/domain/src/init/rule-templates.ts defines exported INIT_SCAFFOLDED_RULE_IDS (comment explains omission of minsky-session-management) and uses it for selection. packages/domain/src/init/rule-templates.test.ts pins exact IDs, asserts omission matches the documented one, and asserts all scaffolded ids are registered.
SC5 — docs/rules/template-system-guide.md:171-177 no longer documents an init --interface flag that does not exist. Met docs/rules/template-system-guide.md lines 172-200 replace incorrect init --interface=... with correct --mcp true/false examples and explicitly state init has no interface flag (and that rules generate does).
SC6 — resolveActiveRules treats disabled as subtractive over the full corpus when neither presets nor enabled is set; with presets/enabled set, they add (intersected with corpus); unknown ids never count toward activeRuleCount. Phase-0 substitution recorded. Met packages/domain/src/rules/rule-selection.ts rewrites resolver to start from corpus, add presets/enabled with corpus intersection, subtract disabled; extensive docblock records Phase-0 substitution. Tests in packages/domain/src/rules/rule-selection.test.ts updated: additive behavior asserted, disabled-only subtractive case added, and unknown id from presets does not inflate the set.

Adoption sweep

Symbol Kind Consumers found Classification Notes
packages/domain/src/compile/compile.ts:minskyCompileTargetsWithGateReport function packages/domain/src/compile/compile.ts:73 — delegated call from minskyCompileTargetsFromPresence, packages/domain/src/compile/compile.test.ts:147 — gate-report unit tests exercising API Adopted Internal adoption via delegating existing API; external callers remain on the original function.

Documentation impact

  • updated-in-pr — This PR changes user-visible behavior (init overwrite merge semantics, compile target gating, id validation) and updates docs accordingly. docs/rules/template-system-guide.md removes the nonexistent init --interface flag, documents --mcp-derived interface, explains re-init merge semantics including refusal, details compile target gating with escapes, and notes id validation and Phase-0 resolver semantics.
    Affected: docs/rules/template-system-guide.md

edobry added a commit that referenced this pull request Sep 4, 2026
…describes

## Summary

When the reviewer concludes `REQUEST_CHANGES` with an empty structured findings channel, the posted
review carries one generic placeholder instead of the issues the model actually described. mt#2828
closed this at the in-loop tool-call boundary, but its own docblock names two paths it cannot reach.
This adds a third post-loop forced pass that covers both.

**Diagnosis first, and verified rather than inherited.** The task had carried "one of two residual
paths fired — not yet verified" since July. Read from the reviewer service's own structured logs on
the Railway deployment that was live at the time (`b6299ad0-…`; the current deployment started at
18:39Z, which is why a default `railway logs` query returns nothing for an 18:20Z review), for
review `5116536812` on PR #3623:

| event | field | value |
| --- | --- | --- |
| `reviewer.empty_findings_recovery_summary` | `applied` | `true` |
| | `concludeReviewGuardRejectionCount` | `0` |
| | `concludeReviewGuardBoundExhausted` | `false` |
| `reviewer.conclude_review_reminder` | `mode` | `post_loop_forced` |
| | `fired_at_turn` | `10` |
| | `gate_branch` | `emitted_no_conclude` |

`rejectionCount: 0` rules out the bound-exhausted path — the in-loop guard never saw a
`conclude_review` call at all. The loop ran to `MAX_TOOL_ROUNDS = 10` without concluding and the
post-loop forced pass supplied the verdict with `tool_choice` pinned to `conclude_review`, so
`submit_finding` was structurally unreachable at the moment the verdict was produced. The whole
output-tool sequence for that review was three calls: `submit_spec_verifications` (the only one the
main loop produced across ten rounds), then both forced passes.

**Magnitude, measured on our own stream** rather than on the task's original "2-in-24h burst"
framing: `measure-recovery-fire-rate.ts --days=21` over 2026-08-14 → 2026-09-04, 543 PRs, 0 errors —
**39 fires across 639 REQUEST_CHANGES rounds (6.1%)**, 1.8% of all 2113 rounds. That MEETS mt#2828's
registered `< 10%` budget, so this is a residual rather than a runaway; it is still ~2/day, and each
one costs the consuming agent a manual prose read against a payload shaped exactly like
`/implement-task` §9's malformed-review bypass condition.

**The upstream nudge already ships and did not prevent it.** `buildRoundBudgetNotice` already
injects *"Emit any findings you are still holding, then call conclude_review"* at two tool-capable
rounds remaining — mt#3547's structural injection, which measurably moved in-loop conclusion 0% →
56% on replay. It produced no finding here, so "tell the model harder" is an exhausted lever and the
remaining fix is a forcing function.

## Key changes

- **`forced-findings-guard.ts`** (new) — pure trigger predicate, mirroring
  `applyEmptyFindingsRecovery`'s condition exactly (last `conclude_review` wins, `REQUEST_CHANGES`,
  zero BLOCKING findings) so what this repairs is precisely what would otherwise reach the mt#2685
  synthesis. Plus the reminder-message builder, which reuses `truncateSummaryForDetails` so the two
  paths embedding the same unbounded model output cannot drift to different budgets.
- **`providers.ts`** — `forceFindings()`, following `forceDocumentationImpact`'s pattern (full
  `ALL_TOOL_DEFINITIONS`, pinned `tool_choice`, shallow-copied messages). Two deliberate differences
  from its siblings: it appends **every** returned call rather than the first, and it applies the
  mt#2863 / mt#3300 resolution-note guard — without that it would be a second `submit_finding`
  emission route bypassing an emission guard, which is the shape of gap this whole task exists to
  close.
- **Keyed on final accumulated state, not on the gate branch.** One predicate therefore covers the
  forced-conclude path and the guard's bound-exhausted fall-through identically — and the fix does
  not depend on the single-review path attribution above generalizing.
- **Corrects a stale comment** that described mt#1471's narrow tools array long after mt#2722
  replaced it with `ALL_TOOL_DEFINITIONS`. Its recorded rejected alternative ("retroactive findings
  would be unanchored from evidence the model never gathered") is a claim about the
  `emitted_nothing` branch; this incident is `emitted_no_conclude`, where nine rounds of evidence
  were gathered and substantive prose written.
- **Retires a can't-fail probe.** `measure-recovery-fire-rate.ts` held its own hard-coded copy of
  the recovery marker sentence, so editing that wording would have made the fire-rate script report
  zero fires with no error — "marker absent" and "pass never fired" are the same observation to it.
  The producer now exports `RECOVERY_FIRE_MARKER_PREFIX` and builds its text from it; the script
  imports it. This is why SC4's reference to mt#2926 is APPENDED rather than a rewrite of the
  opening sentence.

mt#2685's recovery pass is retained as the backstop and still fires when this pass emits nothing.

## Judgment calls

- **Scoped out a second occurrence the spec had been carrying with no owner.** The 2026-07-29
  PR #2392 R4 case (a BLOCKING finding whose text says the issue is resolved) is a different
  mechanism in a different file. Verified against the verbatim finding text that
  `RESOLUTION_NOTE_PATTERN` does not match it — `isResolutionNoteText()` returns `false` on the real
  summary/details pair — and filed as mt#4977. The same round's stale-state re-flagging half is
  mt#4316's class.
- **Shipped a partial recovery knowingly.** The live smoke shows the pinned pass returns exactly ONE
  finding even when the conclusion names two — a property of the `tool_choice` primitive, not of the
  wording. Filed as mt#4979 with the measurement. One real located BLOCKING finding still replaces
  one generic placeholder, but SC1 should be read with that ceiling.
- **Exported `ALL_TOOL_DEFINITIONS`** so the smoke sends the same tools array production sends;
  rebuilding it in the script would have diverged on exactly the axis mem#614 measured.

## Testing

Execution evidence:

AT1 — unit predicate. `bun test --preload ../../tests/setup.ts src/forced-findings-guard.test.ts`:

```
 13 pass
 0 fail
 24 expect() calls
Ran 13 tests across 1 file. [87.00ms]
```

Covers the incident shape (spec verifications + doc impact + REQUEST_CHANGES, zero findings → run),
the bound-exhausted shape, and every skip: BLOCKING finding present, APPROVE, COMMENT, no
`conclude_review`, empty set, NON-BLOCKING/PRE-EXISTING-only, last-conclude-wins in both directions,
and order-independence.

AT2 — pass mechanics and message non-mutation, and AT3's wiring half. Six cases added to
`providers.test.ts` driving the real `callOpenAIWithClient`: fires on the incident shape and lands
two findings at real paths with `tool_choice` pinned to `submit_finding`; does not fire (and makes no
extra API call) when a BLOCKING finding is present or on APPROVE; shallow-copies messages; records
the fall-back when the pass returns nothing; applies the resolution-note guard on this path.

AT4 — full reviewer suite, `cd services/reviewer && bun run test`:

```
 2520 pass
 0 fail
Ran 2520 tests across 97 files. [4.70s]
```

Typecheck 0 errors across 8 projects (`services/reviewer` included, which is where these changes
live); lint 0 errors / 0 warnings across 4385 files.

Negative control — AT2/AT3 wiring. The FULL 65-line forced-findings block was removed from
`providers.ts` (not a one-line tweak) and `providers.test.ts` re-run:

```
(fail) post-loop forced findings pass (mt#2926) > fires on the incident shape and lands the model's own findings in the structured channel
(fail) post-loop forced findings pass (mt#2926) > does not fire when the review already carries a BLOCKING finding — no extra API call
(fail) post-loop forced findings pass (mt#2926) > does not fire on an APPROVE conclusion with zero findings
(fail) post-loop forced findings pass (mt#2926) > shallow-copies messages and ends the forced call with a submit_finding instruction
(fail) post-loop forced findings pass (mt#2926) > records the fall-back when the pass returns no findings, leaving the mt#2685 synthesis to cover it
(fail) post-loop forced findings pass (mt#2926) > applies the mt#2863 resolution-note guard on this path too
 81 pass
 6 fail
```

6/6 of the new wiring cases fail with the wiring gone. Block restored and re-verified byte-identical.

Negative control — the marker-prefix contract. The opening of the synthesized `details` was changed
to `Recovery pass note (mt#2685):` and `empty-findings-recovery.test.ts` re-run:

```
(fail) RECOVERY_FIRE_MARKER_PREFIX > the synthesized finding's details still START with the marker prefix
 13 pass
 1 fail
```

The pin catches exactly the edit that would silently blind the fire-rate script. Reverted and
re-verified.

`[at5-deferred: mt#4980]` — AT5 is a post-deploy fire-rate measurement over a review-volume window;
mt#4980 owns it and carries the 6.1% pre-change baseline to compare against.

## Live verification

`services/reviewer/scripts/smoke-forced-findings.ts` (new, §7a artifact, `OPENAI_API_KEY`-gated,
exits 0/2, writes a structured results file), run against live `gpt-5` with the production
`ALL_TOOL_DEFINITIONS` array and the real reminder builder, on a conclusion naming two file-level

```
=== mt#2926 forced-findings live smoke (gpt-5, 3 attempts) ===
attempt 1: PASS — 1 finding(s) parsed from 1 tool call(s)
    BLOCKING src/hooks/pre-commit.ts:1
attempt 2: PASS — 1 finding(s) parsed from 1 tool call(s)
    BLOCKING src/hooks/pre-commit.ts:1
attempt 3: PASS — 1 finding(s) parsed from 1 tool call(s)
    BLOCKING src/hooks/pre-commit.ts:1

=== Result: 3/3 attempts emitted >= 1 parseable finding ===
```

**3/3 emitted a parseable BLOCKING finding at a real repository path**, not the mt#2685
`(review summary)` sentinel. That is the model-side property no unit test can reach.

Two bounds, stated rather than implied:

1. **One finding per attempt, though the conclusion named two** — the pinned `tool_choice`
   primitive, which `providers.ts` already documents as forcing exactly one call. Filed as mt#4979.
2. **`line: 1` is an artifact of the fixture, not a production finding.** The smoke gives the model
   no diff and no `read_file` results, so it has nothing to anchor a line to. It measures compliance
   with the pin and the shape of the returned call; it does not measure line-number accuracy.

The smoke calls the API directly rather than driving `callOpenAIWithClient`, so it exercises the
model's half only — the six wiring cases above cover the service's half, with the negative control
that proves they can fail.

## Deploy verification

Deploy verification: every changed file is deploy surface (confirmed by running
`isDeploySurfaceFile` from `packages/domain/src/deployment/deploy-surface.ts` over the actual
changed-file list — all five source paths return `true`), so no `[no-deploy-impact]` claim is made.
After merge I will run `deployment_wait-for-latest` for `minsky-reviewer` with `notBefore` set to
the merge timestamp and `expectCommitSha` set to the merge commit, read `buildIdentity`, and assert
the `/health` body's `service` identity rather than the status code. This change adds no new
external-system integration — no new permission, scope, credential, or webhook; the new pass is one
more call to an already-configured provider on an already-provisioned key — so §10's
external-integration live-exercise requirement does not apply, and the deploy-health check plus the
first production `reviewer.forced_findings_pass` log line is the completion signal.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

https://claude.ai/code/session_012HYHcmDv7NuaD6uK7uAvU2

Co-Authored-By: minsky-ai[bot] <minsky-ai[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

authorship/co-authored Co-authored by human and AI agent

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant