You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Should the release exploring run keep a per-release baseline and report only new findings? #15655
Should the release exploring run keep a per-release baseline: diff against the last release, split new from pre-existing findings, and keep stable finding ids?
From the talk
Chapter 10, slides 42-43: an outer-loop run explores each release build like a user and compares it with the last release.
Today and the gap
Absent. A GUI crash-fuzzing run is catalogued as not-a-routine (plugins/autonomy/reference/routines.md:237). The trigger (an autonomy routine) and findings route (/bugs:write) were decided with the live-app exploring skill, Add a live-app exploring skill to the testing plugin, proven on a fixture app; the baseline was not.
Recommendation
Proof of concept (menu priority P1): the exploring skill keeps a baseline per release and reports only new findings as new, with stable ids. Basis: without it, every run refiles known findings.
Alternatives
No baseline, deduplicate at /bugs:write filing time; or a release mode on /bugs:scan holding the baseline.
Done would add
A baseline record and diff step in the exploring skill. Dogfood: two runs against this repository's fixture app across two fixture releases, the second reporting only what changed.
Size: Medium
Complexity: Medium
Uncertainty: Knowable
Flags: none
Blocked by: #15649
I understood this as a request that the release exploring run keep a per-release baseline, diff each run against the last release, report only new findings as new, and give findings stable ids. The issue recommends putting this in the exploring skill. It names two alternatives: deduplicating at /bugs:write filing time, or a release mode on /bugs:scan. The done-when is two dogfood runs against the fixture app across two fixture releases, where the second run reports only what changed.
The exploring skill is tracked in #15649 and is still open, so this work depends on it. I found no merged pull request that already covers this.
Resolved: yes, and it is not a separate skill or item. The comparison with the last release is part of the release pass's own report.
From the talk. Slides 42-43 and the transcript at 1:17:48: on each release an agent runs the whole build "like a real user". The slide-43 report compares every finding with the previous release: "NEW: 3 bugs ... PRE-EXISTING: 47 bugs", and "Each reply keeps its finding id from the full report".
Decision (owner, 2026-11-12: "Let's go with recommendations so far").
The release pass is a whole-map scope on the existing live verification, started by a release routine. No new exploring skill.
That pass's report marks each finding new, pre-existing or gone against the last release's report, keyed by feature-map entry, with an id that stays the same across runs. A first run reports everything as new.
Finding bugs in a build and matching them against the last release change together, so they live in one place.
Declined alternatives. A delta companion skill (a second skill coupled to the first one's finding format); a release mode on /bugs:scan (it hunts by reading code and runs no scanned code unattended); removing duplicates only when /bugs:write files a finding (loses the pre-existing count).
Basis: the talk passages above; her guide (pstack/docs/guide/06-verify-and-ship.md:73 in cursor/plugins), which describes the pass as a swarm of verifiers split by feature-map entry; plugins/verification/skills/confirm/SKILL.md:176-181 and plugins/app-control/skills/map/SKILL.md:206-210, which already hold the fan-out and the every-feature pass.
Activity
github-actions commentedon Nov 12, 2026
Size: Medium
Complexity: Medium
Uncertainty: Knowable
Flags: none
Blocked by: #15649
I understood this as a request that the release exploring run keep a per-release baseline, diff each run against the last release, report only new findings as new, and give findings stable ids. The issue recommends putting this in the exploring skill. It names two alternatives: deduplicating at
/bugs:writefiling time, or a release mode on/bugs:scan. The done-when is two dogfood runs against the fixture app across two fixture releases, where the second run reports only what changed.The exploring skill is tracked in #15649 and is still open, so this work depends on it. I found no merged pull request that already covers this.
Automated triage. A maintainer reviews it.
melodic-automation-lanes commentedon Nov 12, 2026
The triage verdict left this issue at
needs-owner-decision:Reason:
intentkyle-sexton commentedon Nov 13, 2026
Resolved: yes, and it is not a separate skill or item. The comparison with the last release is part of the release pass's own report.
From the talk. Slides 42-43 and the transcript at 1:17:48: on each release an agent runs the whole build "like a real user". The slide-43 report compares every finding with the previous release: "NEW: 3 bugs ... PRE-EXISTING: 47 bugs", and "Each reply keeps its finding id from the full report".
Decision (owner, 2026-11-12: "Let's go with recommendations so far").
Where it is built. Run a whole-map live verification pass on each release and report what is new since the last release, whose interview locks the finding id, the matching rule and where the last report is kept.
Declined alternatives. A delta companion skill (a second skill coupled to the first one's finding format); a release mode on
/bugs:scan(it hunts by reading code and runs no scanned code unattended); removing duplicates only when/bugs:writefiles a finding (loses the pre-existing count).Basis: the talk passages above; her guide (
pstack/docs/guide/06-verify-and-ship.md:73in cursor/plugins), which describes the pass as a swarm of verifiers split by feature-map entry;plugins/verification/skills/confirm/SKILL.md:176-181andplugins/app-control/skills/map/SKILL.md:206-210, which already hold the fan-out and the every-feature pass.kyle-sexton commentedon Nov 13, 2026
Resolved; see the resolution comment.