You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Should verification artifacts be shown in the pull request where a reviewer sees them? #15658
Should verification artifacts (a capture summary, screenshots, video) be linked in the pull request where a human reviewer sees them?
From the talk
Chapter 7, slide 24: review verification artifacts, not diffs.
Today and the gap
Our review lanes' judge reads artifacts, but they are not shown in the pull request, and video is on request only (/playwright:demo-video).
Recommendation
Investigate (menu priority P2), extending /review:explain-change and /playwright:demo-video. Basis: the gap above.
Alternatives
Keep artifacts for the agent judge only; attach video by default.
Done would add
An artifacts section in the pull request body or explainer. Dogfood: how it applies to plugin pull requests here, which change prose and scripts, is open on the map's fog list.
Size: Medium
Complexity: Medium
Uncertainty: Knowable
Flags: intent
Blocked by: none
I understand this as a question on whether verification artifacts (capture summary, screenshots, video) should appear in the pull request for human reviewers, by extending /review:explain-change and /playwright:demo-video. The issue lists two alternatives, and it says how this applies to plugin PRs is still open. No duplicate or merged fix found.
Should artifacts be linked in the PR body or explainer by default?
RECOMMENDED: Yes, as a short artifacts section. Basis: reviewers cannot see what the judge reads today.
Should video be attached by default or stay on request?
RECOMMENDED: Stay on request, with screenshots and the summary by default. Basis: video is costly to produce and rarely applies to prose and script changes.
What should plugin PRs here, which change prose and scripts, show?
RECOMMENDED: The capture summary only, with the section omitted when there are no artifacts. Basis: screenshots and video do not apply to such changes.
To answer, post a comment that starts with this line:
Resolved: already built, with one join added. No new skill and no second place that shows evidence.
From the talk. Slide 24 and the transcript at 1:11:14: "our code review process has now really shrunk into looking at verification artifacts instead. So things like videos, screenshots, diagrams, formal verification."
What already exists.
/source-control:pull-request evidence posts one evidence comment on the pull request, or updates it in place for a new head: the text summary, publishable captures in collapsed sections, and the CI capture artifact link (plugins/source-control/skills/pull-request/reference/evidence.md:5-9).
It posts only after the verifier passed for the pull request's current head: "the evidence comment follows a verifier pass, never stands in for one" (evidence.md:49-51). So an artifact from a run that failed is never posted, and nothing is produced twice for one head.
Unattended, the merge lane posts the same comment after a passing verdict and before merge (evidence.md:11-16).
/playwright:demo-video hands its video to the same step (plugins/playwright/skills/demo-video/SKILL.md:110).
The gap, and the decision (owner, 2026-11-12: "Yes, let's go with recommendations"). In an attended local session nothing runs the evidence step unless someone asks. The ready step will run it once the head is confirmed and there is evidence to post: Post the evidence comment from the ready step once the head is confirmed. That one comment is the home for every kind of artifact: captures, video, and later a formal-verification chart.
Declined. Extending /review:explain-change to show artifacts, and attaching video by default: both would add a second place that shows evidence.
Still open on the map's fog list: what evidence means for this repository's own pull requests, which change prose and scripts and have no app.
Activity
github-actions commentedon Nov 12, 2026
Size: Medium
Complexity: Medium
Uncertainty: Knowable
Flags: intent
Blocked by: none
I understand this as a question on whether verification artifacts (capture summary, screenshots, video) should appear in the pull request for human reviewers, by extending
/review:explain-changeand/playwright:demo-video. The issue lists two alternatives, and it says how this applies to plugin PRs is still open. No duplicate or merged fix found.RECOMMENDED: Yes, as a short artifacts section. Basis: reviewers cannot see what the judge reads today.
RECOMMENDED: Stay on request, with screenshots and the summary by default. Basis: video is costly to produce and rarely applies to prose and script changes.
RECOMMENDED: The capture summary only, with the section omitted when there are no artifacts. Basis: screenshots and video do not apply to such changes.
To answer, post a comment that starts with this line:
Automated triage. A maintainer reviews it.
melodic-automation-lanes commentedon Nov 12, 2026
The triage verdict left this issue at
needs-owner-decision:Reason:
intentkyle-sexton commentedon Nov 13, 2026
Resolved: already built, with one join added. No new skill and no second place that shows evidence.
From the talk. Slide 24 and the transcript at 1:11:14: "our code review process has now really shrunk into looking at verification artifacts instead. So things like videos, screenshots, diagrams, formal verification."
What already exists.
/source-control:pull-request evidenceposts one evidence comment on the pull request, or updates it in place for a new head: the text summary, publishable captures in collapsed sections, and the CI capture artifact link (plugins/source-control/skills/pull-request/reference/evidence.md:5-9).evidence.md:49-51). So an artifact from a run that failed is never posted, and nothing is produced twice for one head.evidence.md:11-16)./playwright:demo-videohands its video to the same step (plugins/playwright/skills/demo-video/SKILL.md:110).The gap, and the decision (owner, 2026-11-12: "Yes, let's go with recommendations"). In an attended local session nothing runs the evidence step unless someone asks. The
readystep will run it once the head is confirmed and there is evidence to post: Post the evidence comment from the ready step once the head is confirmed. That one comment is the home for every kind of artifact: captures, video, and later a formal-verification chart.Declined. Extending
/review:explain-changeto show artifacts, and attaching video by default: both would add a second place that shows evidence.Still open on the map's fog list: what evidence means for this repository's own pull requests, which change prose and scripts and have no app.
kyle-sexton commentedon Nov 13, 2026
Resolved; see the resolution comment.