Skip to content

Rectify: remediation recipe validation immunity#4279

Merged
Trecek merged 12 commits into
developfrom
impl-rectify-remediation-recipe-validation-immunity-20260717-110001
Jul 18, 2026
Merged

Rectify: remediation recipe validation immunity#4279
Trecek merged 12 commits into
developfrom
impl-rectify-remediation-recipe-validation-immunity-20260717-110001

Conversation

@Trecek

@Trecek Trecek commented Jul 17, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Fix the merge_fix_count loop-counter defect in remediation, implementation, and implementation-groups recipes — add missing reset step, route orphaned guards (check_merge_rebase_loop in remediation, init_audit_run in full-audit)
  • Remove all Part A bridge mechanisms — 4 bridge types across 13 test files (~30 instances), including _PART_A_AFFECTED_RECIPES, KNOWN_BROKEN, _PART_A_NON_CONFORMING, and standalone xfail decorators
  • Add xfail governance guard (test_xfail_bridge_policy.py) — AST-walks strict=True xfails requiring #NNNN issue reference in reason strings, with size-capped exemption registry
  • Thread backend pin provenance end-to-end — BackendPinResolution NamedTuple from _resolve_backend_override, origin map through _compute_effective_backend_map → ValidationContext → findings → rejection rendering
  • Wire describe_capability_mismatches() convergence — shared formatter replaces inline formatting in both _preflight.py and rules_backend_compat.py
  • Add two doctor checks — Check 37 (standing backend pin feasibility) and Check 38 (local recipe validity)
  • Update policy text in AGENTS.md §3.1 and make-plan SKILL.md to require issue citation in xfail bridges

Closes #4194

Test plan

  • task test-check passes (29,432+ tests, 0 failures)
  • pre-commit run --all-files clean
  • All bridge mechanisms removed (grep -rn "Part B lands\|_PART_A_AFFECTED_RECIPES\|KNOWN_BROKEN\|_PART_A_NON_CONFORMING" tests/ returns nothing)
  • Recipe diagrams regenerated with current SHA256 hashes
  • Architectural compliance (factory allowlist, import paths, line limits, cascade maps)
  • /audit-impl GO verdict received

🤖 Generated with Claude Code

Trecek
Trecek commented Jul 17, 2026
View reviewed changes

@Trecek Trecek left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AutoSkillit PR Review — Verdict: approved_with_comments

Comment thread src/autoskillit/server/tools/_serve_helpers.py Outdated
Comment thread src/autoskillit/server/tools/tools_recipe.py
Comment thread tests/cli/test_doctor_local_recipes.py Outdated
Trecek
Trecek commented Jul 17, 2026
View reviewed changes

@Trecek Trecek left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AutoSkillit PR Review — Verdict: approved_with_comments

Comment thread tests/cli/test_doctor_standing_pins.py Outdated
Trecek and others added 12 commits July 17, 2026 18:21
@Trecek @claude
fix: add merge_fix_count reset, route orphaned steps, remove all Part…
d43fb60 
… A bridges, add xfail governance guard, and thread backend pin provenance

Phase 1: Add missing reset_merge_fix_counter to remediation/implementation/implementation-groups recipes,
route rebase failures to check_merge_rebase_loop in remediation.yaml, route checkout to init_audit_run
in full-audit.yaml, remove all Part A xfail bridges (13 test files, ~30 instances), flip
test_bundled_recipes_no_missing_reset to assert zero findings, add no-unreachable-steps gate, and
restructure admission_dispatch_agreement with _EXPECTED_INFEASIBLE registry.

Phase 2: Add tests/arch/test_xfail_bridge_policy.py requiring xfail(strict=True) reasons to cite #NNNN
tracking issues, with size-capped exemption registry. Update AGENTS.md and make-plan SKILL.md policy text.
Add catalog row in resolve-review SKILL.md.

Phase 3: Add BackendPinResolution NamedTuple to _resolve_backend_override, thread backend_origin_map
through _compute_effective_backend_map (6 call sites), serve_recipe, load_and_validate,
ValidationContext, and rules_backend_compat. Add origin/remedy fields to RuleFinding and make_finding.
Update _recipe_validation_error_response to surface origin in error messages. Add
describe_capability_mismatches formatter to core. Update _preflight.py rejection envelopes with origin
and remedy. Update tools_execution.py to carry key_path as override source.

Closes #4194

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@Trecek @claude
chore: regenerate recipe diagrams for remediation, implementation, im…
5c17c8b 
…plementation-groups

Hash-stamped diagrams updated to reflect the reset_merge_fix_counter step additions
and orphaned step routing fixes.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@Trecek @claude
fix: update tests for recipe routing changes and filter non-YAML recipes
4299d86 
- Update test_rules_integration_predicate to expect check_merge_rebase_loop
  for remediation.yaml rebase arm (was release_issue_failure)
- Remove rebase arm from remediation-specific terminal-escalation exemption
  in test_all_merge_failure_arms_guarded
- Update test_full_audit_routing_chain to expect checkout → init_audit_run → run_audits
- Filter test_bundled_recipes_no_unreachable_steps to only test recipes
  with existing YAML files in builtin_recipes_dir

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@Trecek @claude
fix: update tests for BackendPinResolution and tuple return from _com…
08d5be6 
…pute_effective_backend_map

All callers of _compute_effective_backend_map now unpack the (dict, origin_map)
tuple. Tests comparing _resolve_backend_override results use .backend instead
of direct string comparison. Arch test counts and allowlists updated for the
new describe_capability_mismatches symbol, shifted line numbers, and
backend_origin_map parameter. Recipe diagrams regenerated with current hashes.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@Trecek @claude
fix: remediate audit findings — wire convergence, add doctor checks, …
87930dc 
…tests

Wire describe_capability_mismatches() into both consumers (_preflight.py
and rules_backend_compat.py), replacing inline formatting with the shared
core function. Add _check_standing_backend_pins_feasibility() and
_check_local_recipe_validity() doctor checks (Checks 37-38), with tests.
Update docs/cli.md counts, cascade map, and architectural allowlists.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@Trecek
fix(review): use 'is not None' for backend_origin_map consistency
5e3b621
@Trecek
fix(review): forward backend_origin_map through validate_from_path
15bda01
@Trecek
fix(review): rename test and simplify fixture to isolate structural v…
a078b9b 
…alidation errors
@Trecek
fix(review): thread backend_origin_map through load_and_validate Prot…
b692cad 
…ocol and repository
@Trecek
fix(review): update test assertions for backend_origin_map propagation
e3d0781
@Trecek @claude
fix(review): tighten standing-pin test assertion from OR to AND
e39d493 
The assertion `"resolve_review" in msg or "codex" in msg` passes if
either substring appears — it doesn't verify the error identifies the
correct pin/backend combination. Changed to AND so both substrings
must be present for a meaningful regression check.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@Trecek @claude
fix: update recipe response ceiling values after rebase
81bc627 
Rebase onto develop changed compiled recipe JSON sizes slightly.
Update the ceiling test expected values to match actual measurements.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@Trecek
Trecek force-pushed the impl-rectify-remediation-recipe-validation-immunity-20260717-110001 branch from bc46d46 to 81bc627 Compare July 18, 2026 01:56
@Trecek
Trecek added this pull request to the merge queue Jul 18, 2026
Merged via the queue into develop with commit 9da15e9 Jul 18, 2026
3 checks passed
@Trecek
Trecek deleted the impl-rectify-remediation-recipe-validation-immunity-20260717-110001 branch July 18, 2026 02:11
This was referenced Jul 18, 2026
Closed
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@Trecek