Contract-Recovery Salvage for Non-Path Enum Verdict Tokens#4316
Merged
Trecek merged 22 commits intoJul 22, 2026
Merged
Conversation
Reproduces the #4305 incident shape (write evidence + path tokens present + enum pattern missing) plus negative controls for the structural soundness gate, ahead of the _headless_recovery.py implementation.
Adds a structural-soundness gate (_parse_single_enum_binding) plus a new recovery stage (_infer_enum_token_from_write_contract) that deterministically synthesizes an enum-typed output token (e.g. verdict = plan) when a conditional-write contract binds exactly one enum value to its write condition and the write actually happened. Runs for all channel confirmations, unlike the existing UNMONITORED-only artifact synthesis, since it derives the token from evidence the agent DID observably produce. Generalizes the CONTRACT_RECOVERY nudge with enum hints: when inference is unsound (multiple values consistent with observed writes), the nudge now asks the session to emit its verdict from the contract's allowed_values instead of silently skipping via nudge_skip_no_hints. Closes the #4305 incident shape: a make-plan session that wrote a valid plan and emitted path tokens but omitted `verdict = plan` is now recoverable in-process without an LLM round-trip.
Rename sensitive-looking logger kwarg 'token' to avoid the ARCH-002 no-sensitive-logger-kwargs false positive, register the new test file in the recipe-cascade file-level entries and layer-boundary allowlist, add _headless_recovery.py's TYPE_CHECKING recipe import to the cross-layer guard inventory, and bump the headless module size budgets to reflect the new inference/nudge code.
…art B) Adds the nine-pair salvage-route property test, Class-1 salvage-continuation test, verify_plan_artifacts callable unit tests, and its contract test. Strengthens make_plan and audit_impl shape tests to pin the new routing. All fail today — verify_plan_artifacts and the salvage steps do not exist yet.
…part B) Verifies capture-sourced plan_parts artifacts exist as non-empty files under a total, deterministic input grammar (bracket/quote/newline/comma rendering variance). Exported via the _cmd_rpc facade and declared in skill_contracts.yaml so recipe salvage steps can route on its verdict.
… call sites (#4305 part B) Class 1 (6 make-plan sites + rectify): new salvage_plan/salvage_plan_phase/ salvage_rectify_plan run_python steps verify captured plan_parts artifacts via verify_plan_artifacts and route salvaged onward to the plan step's own success destination, unsalvageable to its own on_failure destination. Class 2 (dry_walkthrough): new retry_walkthrough bounded one-hop re-invoke, mirroring the retry_worktree precedent. Class 3 (audit_impl): on_context_limit now routes through the existing check_audit_remediation_loop guard instead of the decorative register_clone_failure alias of on_failure. Fixes the salvage-continuation test to read the plan step's success destination from on_success when on_result predicate routing is absent (rectify uses on_success, not verdict-based on_result). Recompiled contract cards and JSON mirrors via task regen-contracts && task compile-recipes.
Fixes 13 test failures surfaced by the nine-site salvage routing change: byte-budget overage in the codex delivery envelope (omit empty skeleton summaries), five stale recipe-diagram hashes, a stdlib re import that should use the regex package, a missing on_rate_limit on the new retry_walkthrough step, an unwired remediation_path handoff reference, a verdict-allowed-values test that did not scan callable_contracts, and a missing Context Limit Behavior section on rectify SKILL.md now that it has a real on_context_limit route. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
… part C) Derives a salvage-route requirement directly from skill contract data (non-empty expected_output_patterns + not read_only) rather than a hand-maintained site list, mirroring the verdict-context-precondition / run-skill-missing-context-limit precedent patterns. Severity is WARNING, not ERROR as originally scoped: the contract-derived eligibility predicate matches far more bundled-recipe steps than the nine sites audited and fixed in the prior part (any non-read-only skill with expected_output_patterns qualifies). Promoting straight to ERROR today would flip valid=False for roughly a dozen bundled recipes with pre-existing, unremediated gaps, and this codebase's own governance (test_error_severity_rules_have_no_dispatch_ready_exemptions) forbids exempting ERROR rules by design. Ship at WARNING now; promote once a follow-up remediation pass closes the remaining gaps (see module docstring). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…#4305 part C) Replace _CONTEXT_LIMIT_COMPLIANT_RECIPES and _RATE_LIMIT_COMPLIANT_RECIPES (recipe-level allowlists hidden behind untracked imperative pytest.xfail() calls) with the existing per-step CONTEXT_LIMIT_EXEMPT_STEPS / RATE_LIMIT_EXEMPT_STEPS registries, populated with the real gaps surfaced once every bundled recipe runs the check unconditionally. Both registries are now capped and require a tracking citation per non-empty entry (mirrors test_rule_severity_consistency.py's allowlist governance pattern), enforced by four new companion tests. Fixes test_rules_rate_limit_parity.py, which imported the now-deleted _RATE_LIMIT_COMPLIANT_RECIPES — it now computes expected missing steps directly from the raw recipe instead of depending on the allowlist. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…cy (#4305 part C) _collect_xfail_strict_true_nodes only visits decorator and pytest.param(marks=...) call sites, both of which carry a strict= keyword. The imperative form (pytest.xfail(reason) as a bare statement inside a test body) raises immediately at call time, has no strict= keyword, and was invisible to this policy — exactly the blind spot that hid the untracked context-limit-compliance exemption in test_bundled_recipes_behavioral_properties.py (dismantled in a prior commit). Adds a sibling collector for the Expr-statement call shape plus a tree-sweep test requiring every imperative xfail's reason to cite a #NNNN tracking issue, and synthetic-AST tests pinning the collector's fire/no-fire behavior. Sweeping the tree surfaced two further pre-existing imperative xfails, both guarded by their own bounded known-violations registries and unrelated to #4305's scope; exempted with rationale rather than fabricating tracking issues for unrelated bugs. Also adds a dispatch-ready regression test scoped to the nine audited contract-recovery call sites (dual coverage alongside the existing behavioral- property structural test) rather than asserting zero findings across all bundled recipes, since the new rule's contract-derived predicate also flags steps outside the audited set that are not yet remediated (see rules_contract_recovery.py). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…ery vocabulary (#4305 part C) Extends test_routing_completeness.py so the run_skill docstring's routing table (tools_execution.py) is checked against the same contract_recovery -> on_context_limit + has_progress_evidence terms already asserted against the orchestrator prompt. The third prose copy (SKILL.md) is already covered by tests/contracts/test_sous_chef_routing.py. Pins all three routing-vocabulary copies so silent drift in any one fails a test instead of surfacing as a live routing gap. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…lists after #4305 part C growth The new contract-recovery-requires-salvage-route rule (part C) and its prerequisite salvage-route wiring (part B) grew the remediation recipe and recipe/rules/ past several measured/pinned ceilings: - RESPONSE_BACKSTOP_EXEMPTION_REGISTRY (load_recipe/open_kitchen) measured ceiling bumped from 188_000 to 195_000 chars/bytes, re-measured today; CODEX_TOOL_OUTPUT_TOKEN_LIMIT and its fixture/test pins recompute from the new ceiling. - recipe/rules/ subpackage file-count exemption bumped 54 -> 55 for the new rules_contract_recovery.py module. - pyright suppression allowlist line number in recipe/__init__.py corrected (283 -> 286) after new import lines shifted it. Also fixes a latent surface-parity bug in test_serve_idempotence.py: for a recipe this close to the claude-code delivery bound, open_kitchen's pre-envelope payload (extra routing metadata via build_open_kitchen_recipe_payload) can cross the bound independently of load_recipe's slightly smaller payload, so one surface may serve the bounded step-flow-skeleton envelope while the other still serves full content. Both tests now fall back to comparing the recipe-derived routing fields (post_prune_step_names/post_prune_routing_edges) that are populated identically in either representation. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…e-test-audit/audit-impl (#4305 remediation part A) T-A1/T-A2 from the Part A remediation plan: fail today because skill_contracts.yaml has no allowed_values for these three skills' verdict outputs, so enum recovery inference/nudge machinery has zero live surface.
…-note verbatim-substitution instruction (#4305 remediation part A) Adds allowed_values to skill_contracts.yaml for validate-audit/validate-test-audit (activates deterministic enum inference) and audit-impl (activates the enum nudge). Closes the resulting skill-result-routing-gap ERROR coupling by adding an explicit, behavior-preserving NO GO route to the audit_impl step in implementation.yaml, remediation.yaml, research.yaml, research-implement.yaml, and implementation-groups.yaml (merge-prs.yaml already routes NO GO explicitly and is unchanged). Also appends the verbatim-substitution instruction to all seven salvage-step notes so orchestrators pass context.plan_parts through to the callable unmodified. Regenerates contract cards, compiled recipe JSON, and diagram hash markers (route destinations are unchanged, so diagram bodies are untouched).
… NO GO The regex's value group used \w+ which stops at the space in "NO GO", capturing only "NO". This caused on-result-values-in-allowed-values to flag every audit_impl NO GO route as unrecognized, cascading to 83 test failures across recipe validation. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The regex fix changed recipe validation results (NO GO routes now parse correctly), increasing the remediation recipe content size by 547 bytes. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…vacuous-true test
…ironment verdict outputs
…etup-environment verdict outputs" This reverts commit 4804367.
…o avoid vacuous-true test" This reverts commit d3b2fff.
Trecek
deleted the
non-path-verdict-token-unrecoverable-make-plan-lacks-on-cont/4305-2
branch
July 22, 2026 15:38
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This change makes contract recovery aware of non-path enum verdict tokens, adds deterministic salvage routing for artifact-producing recipe steps, and enforces those guarantees at authoring time. It also completes the contract metadata and recipe-note remediation needed to activate enum inference and preserve captured plan-part values verbatim.
Individual Group Plans
Group 1: Rectify: Contract-Recovery Immunity for Non-Path Enum Tokens
The recovery layer becomes contract-aware: deterministic enum inference reconstructs mechanically implied verdicts from contract metadata and observed writes, while generalized enum hints let resumed sessions emit verdicts when inference would be unsound.
Group 2: Rectify: Recipe-Layer Salvage Routes for Contract-Recovery-Capable Steps
Contract-recovery-capable recipe steps receive bounded, non-destructive
on_context_limitsalvage routes. Plan-producing steps reuse captured artifact references through deterministic validation and retain their original failure destinations as a hard floor.Group 3: Rectify: Authoring-Time Enforcement — Contract-Recovery Salvage Guarantees
A contract-derived semantic rule requires distinct salvage routes for capable skills, the allowlist-based compliance gap is replaced with assert-by-default coverage, and the xfail policy guard is extended to catch imperative
pytest.xfail()calls.Group 4: Implementation Plan: Contract-Recovery Salvage Remediation — Enum Contract Metadata + Salvage-Note Verbatim Substitution
Contract metadata gains the missing allowed verdict values, salvage notes explicitly require verbatim substitution of captured
plan_parts, and affected audit routes gain explicit behavior-preservingNO GOconditions.Closes #4305
Implementation Plan
Plan files:
/home/talon/projects/autoskillit-runs/remediation-20260721-201600-381418/.autoskillit/temp/rectify/rectify_contract_recovery_salvage_2026-07-21_202404_part_a.md/home/talon/projects/autoskillit-runs/remediation-20260721-201600-381418/.autoskillit/temp/rectify/rectify_contract_recovery_salvage_2026-07-21_202404_part_b.md/home/talon/projects/autoskillit-runs/remediation-20260721-201600-381418/.autoskillit/temp/rectify/rectify_contract_recovery_salvage_2026-07-21_202404_part_c.md/home/talon/projects/autoskillit-runs/remediation-20260721-201600-381418/.autoskillit/temp/make-plan/contract_recovery_salvage_remediation_plan_2026-07-22_022055_part_a.md🤖 Generated with Claude Code via AutoSkillit