Repository navigation
Conversation
[CCR-20261030-001] · reviewer · anthropic/claude-sonnet-5-5→? · high · schema:2Initial review: PR #375 (work item #373)Verdict: REVIEW_PASSED_AWAITING_CI. No finding. The review is complete for head Scope
FindingsNone. Checked:
Security triageNot a sensitive change: the security audit was skipped after triage. The runtime plan has Execution (on
|
| Command | Result |
|---|---|
python3 -m unittest tests.test_verification_plan |
exit 0, 27 tests OK |
python3 -m unittest discover -s tests |
exit 0, 2125 tests OK, 1 skipped |
python3 scripts/validate-package.py |
exit 0, VALID: 16 skills (7 cycle, 9 supporting) |
python3 -m unittest (the bare configured command) was not run: it discovers 0 tests in this repository, as the PR description notes, and the discovery command above covers the suite.
Acceptance criteria (tests.version 2)
| ID | Criterion | Evidence | State |
|---|---|---|---|
| AC-1 | v2 non-object evidence/attempts raises VerificationError naming the check, never AttributeError |
test_v2_rejects_string_evidence_and_attempts_with_check_name, verification_plan.py:345-352 |
met |
| AC-2 | A string is not coerced to {"summary": ...} |
same test; the entry is rejected, not converted | met |
| AC-3 | v1 with a plan keeps invalid legacy evidence, has a string-entries test; v1 without a plan unchanged |
test_v1_string_evidence_rejects_with_plan_and_stays_unchanged_without_one |
met |
| AC-4 | The rejection reaches the cycle as a controlled outcome, never verified |
test_string_check_entries_reach_cycle_as_not_verified |
met |
| AC-5 | Regression tests cover v2 string evidence and attempts, and v1 string entries with a plan | the three new tests, 2125-test suite green | met |
| AC-6 | Skill or stage prompt and the owning docs/ page describe the evidence object shape |
skills/cc-verify/SKILL.md:31-33, docs/verification.md:239-242; by inspection, plus the package validator |
met (inspection) |
{"version": 2, "ran": true, "passed": true, "conclusion": "verified_with_reservations", "checks": [
{"id": "configured-test", "command": "python3 -m unittest discover -s tests", "criteria": ["AC-1", "AC-2", "AC-3", "AC-4", "AC-5"], "level": "test", "head_sha": "83335639dd5798a7a9b739f2b3b9a1422049ff8d", "result": "pass", "evidence": [{"exit_code": 0, "executed": 2125}], "cause": "", "attempts": []},
{"id": "package-validator", "command": "python3 scripts/validate-package.py", "criteria": ["AC-6"], "level": "static", "head_sha": "83335639dd5798a7a9b739f2b3b9a1422049ff8d", "result": "pass", "evidence": [{"exit_code": 0, "executed": null}], "cause": "", "attempts": []}
]}Unverified
- The merge head
875e0bf5…was not built or tested locally: this stage has no remote and cannot fetch it. The merge bringsmainchanges I did not see. The tests above passed on its first parent's tree only. The exact-head CI below is the evidence for the merged tree. - AC-6 is checked by reading the text; no test asserts the documentation wording.
CI (head 875e0bf5e6a2b56a05db386b7c8b49d250242cb0, policy parallel, read once)
| Check | State |
|---|---|
| Skills, manifests, and Bash installer | pending |
| Windows test shard (windows-1) | pending |
| Windows test shard (windows-2) | pending |
| Windows test shard (windows-3) | pending |
| Windows test shard (windows-4) | pending |
Not waited on. The previous head's run had 4 of 5 passed and windows-4 in progress.
Residual risks
- The merge from
mainis covered only by the pending CI. - The check-id-named message is built from
check.id(agent-supplied string) and is raised as an exception message only; it is not rendered to a command or a shell.
ORCHESTRATION_RESULT (JSON)
ORCHESTRATION_RESULT
{
"skill": "cc-initial-review",
"status": "REVIEW_PASSED_AWAITING_CI",
"issue_provider": "github",
"issue_id": "373",
"code_host": "github",
"change_request_id": "375",
"change_request_url": "#375",
"pr_number": 375,
"issue_number": 373,
"head_sha": "875e0bf5e6a2b56a05db386b7c8b49d250242cb0",
"reviewed_head_sha": "875e0bf5e6a2b56a05db386b7c8b49d250242cb0",
"security_audit_ran": false,
"review_run_id": "CCR-20261030-001",
"plan_id": "plan-cycle-2671a0484a864aca8f331f3b40e027cb",
"revision": 4,
"digest": "1a7626b6aa3cee35d6c8f45915391fa0ad9bc9e7872acc0a7c1e3edfda539089",
"reviewer": {
"profile": "reviewer",
"provider": "anthropic",
"model_requested": "claude-sonnet-5-5",
"model_resolved": null,
"effort": "high"
},
"findings": [],
"blocking_findings": [],
"checks": {"head_sha": "875e0bf5e6a2b56a05db386b7c8b49d250242cb0", "passed": 0, "failed": 0, "pending": 5},
"tests": {"version": 2, "conclusion": "verified_with_reservations", "passed": true, "checks": [
{"id": "configured-test", "criteria": ["AC-1", "AC-2", "AC-3", "AC-4", "AC-5"], "head_sha": "83335639dd5798a7a9b739f2b3b9a1422049ff8d", "result": "pass"},
{"id": "package-validator", "criteria": ["AC-6"], "head_sha": "83335639dd5798a7a9b739f2b3b9a1422049ff8d", "result": "pass"}
]},
"blocking": false
}
END_ORCHESTRATION_RESULT
[CCR-20261030-002] · reviewer · anthropic/claude-sonnet-5-5→? · high · schema:2Initial review: PR #375 (work item #373)Verdict: REVIEW_PASSED_AWAITING_CI. No finding. The review is complete for head Scope
FindingsNone. Checked:
Security triageNot a sensitive change: the security audit was skipped after triage. The runtime plan has Execution (on
|
| Command | Result |
|---|---|
python3 -m unittest tests.test_verification_plan |
exit 0, 31 tests OK |
python3 -m unittest discover -s tests |
exit 0, 2129 tests OK, 1 skipped |
python3 scripts/validate-package.py |
exit 0, VALID: 16 skills (7 cycle, 9 supporting) |
python3 -m unittest |
exit 0, 0 tests discovered (no tests at the repository root; the discovery command above runs the suite) |
Acceptance criteria (tests.version 2)
| ID | Criterion | Evidence | State |
|---|---|---|---|
| AC-1 | v2 non-object evidence/attempts raises VerificationError naming the check, never AttributeError |
test_v2_rejects_string_evidence_and_attempts_with_check_name, verification_plan.py:345-352 |
met |
| AC-2 | A string is not coerced to {"summary": ...} |
same test; the entry is rejected | met |
| AC-3 | v1 with a plan keeps invalid legacy evidence and gains a string-entries test; v1 without a plan unchanged |
test_v1_string_evidence_rejects_with_plan_and_stays_unchanged_without_one |
met |
| AC-4 | The rejection reaches the cycle as a controlled outcome, never verified |
test_string_check_entries_reach_cycle_as_not_verified |
met |
| AC-5 | Regression tests for v2 string evidence and attempts, and v1 string entries with a plan | the three new tests; 2129-test suite green | met |
| AC-6 | Skill or stage prompt and owning docs/ page describe the evidence object shape |
skills/cc-verify/SKILL.md:31-33, docs/verification.md:239-242; by inspection plus the package validator |
met (inspection) |
Unverified
- AC-6 is checked by reading the text; no test asserts the documentation wording.
CI (head 875e0bf5e6a2b56a05db386b7c8b49d250242cb0, policy parallel, read once, not waited on)
| Check | State |
|---|---|
| Skills, manifests, and Bash installer | pass |
| Windows test shard (windows-2) | pass |
| Windows test shard (windows-4) | pass |
| Windows test shard (windows-1) | pending |
| Windows test shard (windows-3) | pending |
Residual risks
- Two Windows shards are still pending; the CI gate decides on them.
- The error message embeds
check.id(agent-supplied string) and is raised only as an exception message; it is not rendered to a command or shell.
ORCHESTRATION_RESULT (JSON)
ORCHESTRATION_RESULT
{
"skill": "cc-initial-review",
"status": "REVIEW_PASSED_AWAITING_CI",
"issue_provider": "github",
"issue_id": "373",
"code_host": "github",
"change_request_id": "375",
"change_request_url": "#375",
"pr_number": 375,
"issue_number": 373,
"head_sha": "875e0bf5e6a2b56a05db386b7c8b49d250242cb0",
"reviewed_head_sha": "875e0bf5e6a2b56a05db386b7c8b49d250242cb0",
"security_audit_ran": false,
"review_run_id": "CCR-20261030-002",
"plan_id": "plan-cycle-2671a0484a864aca8f331f3b40e027cb",
"revision": 8,
"digest": "906cc0655f303e2062262699b3d0e40343588f7cd79a874eaf0a06eeee283c29",
"reviewer": {
"profile": "reviewer",
"provider": "anthropic",
"model_requested": "claude-sonnet-5-5",
"model_resolved": null,
"effort": "high"
},
"findings": [],
"blocking_findings": [],
"checks": {"head_sha": "875e0bf5e6a2b56a05db386b7c8b49d250242cb0", "passed": 3, "failed": 0, "pending": 2},
"tests": {"version": 2, "conclusion": "verified_with_reservations", "passed": true, "checks": [
{"id": "configured-test", "criteria": ["AC-1", "AC-2", "AC-3", "AC-4", "AC-5"], "head_sha": "875e0bf5e6a2b56a05db386b7c8b49d250242cb0", "result": "pass"},
{"id": "package-validator", "criteria": ["AC-6"], "head_sha": "875e0bf5e6a2b56a05db386b7c8b49d250242cb0", "result": "pass"}
]},
"blocking": false
}
END_ORCHESTRATION_RESULT
[CCR-20261031-001] · reviewer · anthropic/claude-sonnet-5-5→? · high · schema:2Initial review: PR #375 (work item 373)Status: REVIEW_PASSED_AWAITING_CI: the review is complete for head ScopeAccumulated diff FindingsNone. Acceptance criteria
Nested history entries with string evidence are also rejected as Recovered attempt verificationRecovered attempt Security triageThe deterministic rule did not fire: no changed path matches the mandatory minimum or a project rule, and the PR has no labels. My own reading found no sensitive surface; the change is input validation of an internal verification report. The runtime plan has Commands executed (local, at the reviewed head)
Unverified: none. CI (policy: parallel; read once, not awaited)All six checks completed with SUCCESS on this head: Skills, manifests, and Bash installer; Windows test shards windows-1 to windows-4; PowerShell installer. Residual riskLow. The validator accepts any VerdictNo findings. Review complete for |
Summary
Closes #373
Reject malformed check-result entries with a controlled verification error. String values in
evidenceorattemptsare not converted into evidence.Diagnosis
normalize_testsaccessed.get()on every v2 evidence item. Reproduction: a v2 check with"evidence": ["text"]raisedAttributeError: 'str' object has no attribute 'get'inscripts/verification_plan.py. The cause matches the issue. The invariant is that each check's evidence and attempt entries are objects before normalization.run_cyclealready convertsVerificationErrorinto a controllednot_verifiedoutcome. Related contract work: #290; widened issue search found no other item for this malformed-entry failure.The fix validates the
evidenceandattemptsarrays and raisesVerificationErrornaming the check, field, entry index, and entry type. The v1 adapter keeps itsinvalid legacy evidencerejection when a plan is present; v1 reports without a plan remain unchanged.Work units
WU-1scripts/verification_plan.py,tests/test_verification_plan.py,skills/cc-verify/SKILL.md,docs/verification.mdpython3 -m unittest discover -s tests;python3 scripts/validate-package.py;python3 -m unittestVerification (tests.version 2)
The configured check covers the acceptance criteria in their supplied order. Local verification ran on Linux against the exact PR head. GitHub CI runs independently on the pushed head.
{ "version": 2, "ran": true, "passed": true, "conclusion": "verified_with_reservations", "boundary": "required", "checks": [ { "id": "configured-test", "command": "python3 -m unittest discover -s tests", "executor": "claude", "profile": "cheap_coder", "context": "project", "required": true, "criteria": ["AC-1", "AC-2", "AC-3", "AC-4", "AC-5"], "level": "test", "provenance": "project", "reason": "configured test command", "head_sha": "83335639dd5798a7a9b739f2b3b9a1422049ff8d", "result": "pass", "evidence": [{"exit_code": 0, "executed": 2125}], "cause": "", "attempts": [] }, { "id": "package-validator", "command": "python3 scripts/validate-package.py", "executor": "claude", "profile": "cheap_coder", "context": "project", "required": true, "criteria": ["AC-6"], "level": "static", "provenance": "project", "reason": "configured verification command", "head_sha": "83335639dd5798a7a9b739f2b3b9a1422049ff8d", "result": "pass", "evidence": [{"exit_code": 0, "executed": null}], "cause": "", "attempts": [] }, { "id": "full-unittest", "command": "python3 -m unittest", "executor": "claude", "profile": "cheap_coder", "context": "project", "required": true, "criteria": [], "level": "test", "provenance": "project", "reason": "configured verification command", "head_sha": "83335639dd5798a7a9b739f2b3b9a1422049ff8d", "result": "pass", "evidence": [{"exit_code": 0, "executed": 0}], "cause": "", "attempts": [] } ] }The first discovery run completed 2,125 tests (1 skipped); it was rerun against this exact head and passed with the same count. Package validation passed. The bare
python3 -m unittestcommand exited successfully but discovered 0 tests; the configured discovery command above ran the suite. BasedPyright reported 0 errors and 99 warnings, and exited 1 because warnings produce a non-zero status; the warning baseline was not compared.Checks of pushed head
Head SHA:
83335639dd5798a7a9b739f2b3b9a1422049ff8dThis is the single post-push check read. The two pending Windows shards remain pending; this stage did not wait for them.
Stage receipt