What happened
The code agent on PR #4620 bumped the boost workspace to Backstage 1.54.4, including two 0.x packages with minor version increments: @backstage/frontend-plugin-api (^0.17.2 → ^0.18.0) and @backstage/repo-tools (^0.17.3 → ^0.19.0). The issue #4617 instructions included "read release notes, search source for affected APIs" as a verification step, and the code agent's tests/build/tsc all passed — suggesting it performed this verification. However, the PR body only documented that "All tests pass. Build and prettier checks pass" without mentioning upstream changelog review.
The review agent then flagged a low-severity semver-risk finding: "Under semver 0.x conventions, minor version increments may introduce breaking changes... Verify the upstream changelogs for breaking API changes." The finding was valid but generic — it asked the human reviewer to verify something the code agent likely already checked but didn't document. The human reviewer approved 24 minutes later without comments.
What could go better
The review agent's semver-risk finding is predictable: it will recur on every Backstage bump that touches 0.x packages, because the code agent does not document its changelog verification. This creates a pattern where (1) the code agent checks changelogs as instructed, (2) the review agent can't see evidence of this check, (3) the review agent flags a generic "verify changelogs" finding, and (4) the human must independently verify or trust the code agent's work.
If the code agent documented its changelog findings in the PR body (e.g., which 0.x packages had minor bumps, whether breaking changes were found, and what the relevant changelog entries said), the review agent could verify the documentation rather than flagging a generic risk. This would also give human reviewers explicit evidence, potentially reducing the review-to-merge gap.
Confidence: moderate. I'm confident the pattern will recur (any Backstage bump touching 0.x packages will trigger it), but I'm uncertain whether the human reviewer's 24-minute gap was actually spent checking changelogs or was simply availability lag.
Proposed change
Add a paragraph to .fullsend/AGENTS.md under or near the existing changeset bump-level guidance. The new instruction should state that when performing a Backstage version bump (yarn backstage-cli versions:bump), if any @backstage/* dependency has a 0.x version with a minor version change, the code agent must:
- Check the upstream Backstage changelog or release notes for those specific packages.
- Document in the PR body which 0.x packages had minor bumps and whether breaking changes were identified.
- If breaking changes are found, adjust the changeset severity from
patch to minor for affected consuming packages and note the required migration steps.
Example PR body addition:
### 0.x dependency changes
- @backstage/frontend-plugin-api ^0.17.2 → ^0.18.0: Checked changelog — no breaking changes affecting boost plugins (new optional props added to `useRouteRef`).
- @backstage/repo-tools ^0.17.3 → ^0.19.0: Checked changelog — `@useoptic/openapi-utilities` dependency removed upstream, no API changes affecting boost.
Alternatively, if the rhdh-backstage-upgrade skill exists in the agents repo (fullsend-ai/agents), this instruction could be added there as an additional step after the existing "read release notes" step.
Validation criteria
On the next 3 Backstage version bump PRs for any workspace in this repo: (1) the PR body includes a section documenting 0.x changelog verification when applicable, and (2) the review agent does not flag a generic semver-risk finding for packages whose changelogs were already documented in the PR body.
Generated by retro agent from #4620
What happened
The code agent on PR #4620 bumped the boost workspace to Backstage 1.54.4, including two 0.x packages with minor version increments:
@backstage/frontend-plugin-api(^0.17.2 → ^0.18.0) and@backstage/repo-tools(^0.17.3 → ^0.19.0). The issue #4617 instructions included "read release notes, search source for affected APIs" as a verification step, and the code agent's tests/build/tsc all passed — suggesting it performed this verification. However, the PR body only documented that "All tests pass. Build and prettier checks pass" without mentioning upstream changelog review.The review agent then flagged a low-severity semver-risk finding: "Under semver 0.x conventions, minor version increments may introduce breaking changes... Verify the upstream changelogs for breaking API changes." The finding was valid but generic — it asked the human reviewer to verify something the code agent likely already checked but didn't document. The human reviewer approved 24 minutes later without comments.
What could go better
The review agent's semver-risk finding is predictable: it will recur on every Backstage bump that touches 0.x packages, because the code agent does not document its changelog verification. This creates a pattern where (1) the code agent checks changelogs as instructed, (2) the review agent can't see evidence of this check, (3) the review agent flags a generic "verify changelogs" finding, and (4) the human must independently verify or trust the code agent's work.
If the code agent documented its changelog findings in the PR body (e.g., which 0.x packages had minor bumps, whether breaking changes were found, and what the relevant changelog entries said), the review agent could verify the documentation rather than flagging a generic risk. This would also give human reviewers explicit evidence, potentially reducing the review-to-merge gap.
Confidence: moderate. I'm confident the pattern will recur (any Backstage bump touching 0.x packages will trigger it), but I'm uncertain whether the human reviewer's 24-minute gap was actually spent checking changelogs or was simply availability lag.
Proposed change
Add a paragraph to
.fullsend/AGENTS.mdunder or near the existing changeset bump-level guidance. The new instruction should state that when performing a Backstage version bump (yarn backstage-cli versions:bump), if any@backstage/*dependency has a 0.x version with a minor version change, the code agent must:patchtominorfor affected consuming packages and note the required migration steps.Example PR body addition:
Alternatively, if the
rhdh-backstage-upgradeskill exists in the agents repo (fullsend-ai/agents), this instruction could be added there as an additional step after the existing "read release notes" step.Validation criteria
On the next 3 Backstage version bump PRs for any workspace in this repo: (1) the PR body includes a section documenting 0.x changelog verification when applicable, and (2) the review agent does not flag a generic semver-risk finding for packages whose changelogs were already documented in the PR body.
Generated by retro agent from #4620