What happened
On PR #4574, the code agent authored both an openspec spec file (openspec/specs/ogx-entity-provider/spec.md) and the implementation (OgxModelEntityProvider.ts) in the same commit. The spec stated that invalid caData should "still apply the configured caData and issue the request" (spec line 119), but the implementation returned undefined on PEM validation failure (code line 200), falling back to system CAs.
This spec-implementation divergence was the #1 finding across all 6 review rounds, flagged as HIGH severity in the first review (run 34520203260) and again in the second review (run 34524473245). The fix agent attempted to resolve it in iteration 1, but gabemontero later deliberately chose the fail-open behavior (commit 4d0d2db) without updating the spec, so the divergence persisted through all subsequent review rounds and into the merge.
The existing AGENTS.md has openspec guidance on scenario step discipline (lines 48–81) and cancellation checklists (lines 83–105), but no guidance on verifying that implementation matches spec before committing.
What could go better
A self-verification step — comparing each behavioral contract in the spec against the actual code before committing — would have caught this inconsistency at the source. The code agent would have either aligned the code with the spec or updated the spec to match its implementation choice, preventing the finding from appearing in the first place.
This was the most persistent and costly finding: it appeared in every review round, required fix agent work, and was ultimately resolved by human intervention rather than the automated workflow. Preventing it at the code-agent stage would have reduced the review churn significantly.
Confidence: moderate. The code agent may have made a deliberate implementation choice that differed from the spec text it wrote — a verification step would flag the inconsistency but cannot guarantee the agent resolves it correctly. However, even flagging it would prompt reconciliation before submission. The boost AGENTS.md already contains detailed openspec guidance that the code agent follows, so adding a verification step to the same section is the natural location.
Related: fullsend-ai/fullsend#3497 proposes general documentation-vs-code verification at the code agent definition level. This proposal adds repo-specific guidance for the openspec pattern used in the boost workspace.
Proposed change
In workspaces/boost/AGENTS.md, at the end of the "Specification-driven development" section (after line 81, before the cancellation subsection), add:
### Spec-implementation consistency check
After implementing code that has a corresponding openspec spec file
(under `openspec/specs/`), verify consistency before committing:
1. Re-read each behavioral contract in the spec (GIVEN/WHEN/THEN
scenarios, prose requirements) and confirm the implementation
matches. Pay special attention to error paths, fallback behavior,
and edge cases — these are the most common sources of divergence.
2. If the implementation deliberately diverges from the spec (e.g., a
different error-handling strategy), update the spec to match the
actual behavior. The spec is the behavior source of truth and must
not describe behavior the code does not implement.
3. If creating a new spec alongside new code, write the spec after
the implementation is finalized, or re-read and reconcile the spec
against the final code before committing.
Validation criteria
The next 3 code agent PRs that modify both an openspec/specs/ file and its corresponding plugin source should not have spec-implementation divergence flagged by the review agent in the first review round. Specifically, no HIGH-severity findings about spec-code contradiction should appear for PRs that touch both spec and implementation files.
Generated by retro agent from #4574
What happened
On PR #4574, the code agent authored both an openspec spec file (
openspec/specs/ogx-entity-provider/spec.md) and the implementation (OgxModelEntityProvider.ts) in the same commit. The spec stated that invalidcaDatashould "still apply the configured caData and issue the request" (spec line 119), but the implementation returnedundefinedon PEM validation failure (code line 200), falling back to system CAs.This spec-implementation divergence was the #1 finding across all 6 review rounds, flagged as HIGH severity in the first review (run 34520203260) and again in the second review (run 34524473245). The fix agent attempted to resolve it in iteration 1, but gabemontero later deliberately chose the fail-open behavior (commit
4d0d2db) without updating the spec, so the divergence persisted through all subsequent review rounds and into the merge.The existing AGENTS.md has openspec guidance on scenario step discipline (lines 48–81) and cancellation checklists (lines 83–105), but no guidance on verifying that implementation matches spec before committing.
What could go better
A self-verification step — comparing each behavioral contract in the spec against the actual code before committing — would have caught this inconsistency at the source. The code agent would have either aligned the code with the spec or updated the spec to match its implementation choice, preventing the finding from appearing in the first place.
This was the most persistent and costly finding: it appeared in every review round, required fix agent work, and was ultimately resolved by human intervention rather than the automated workflow. Preventing it at the code-agent stage would have reduced the review churn significantly.
Confidence: moderate. The code agent may have made a deliberate implementation choice that differed from the spec text it wrote — a verification step would flag the inconsistency but cannot guarantee the agent resolves it correctly. However, even flagging it would prompt reconciliation before submission. The boost AGENTS.md already contains detailed openspec guidance that the code agent follows, so adding a verification step to the same section is the natural location.
Related: fullsend-ai/fullsend#3497 proposes general documentation-vs-code verification at the code agent definition level. This proposal adds repo-specific guidance for the openspec pattern used in the boost workspace.
Proposed change
In
workspaces/boost/AGENTS.md, at the end of the "Specification-driven development" section (after line 81, before the cancellation subsection), add:Validation criteria
The next 3 code agent PRs that modify both an
openspec/specs/file and its corresponding plugin source should not have spec-implementation divergence flagged by the review agent in the first review round. Specifically, no HIGH-severity findings about spec-code contradiction should appear for PRs that touch both spec and implementation files.Generated by retro agent from #4574