Skip to content

Add the manual-dispatch workflow for the whole-map live verification pass #16564

Description

@kyle-sexton

Split from #15649 at intake's recommendation: the on-demand workflow is host work, since an unattended lane run cannot dispatch it or use the pr-verify key's sandbox. #15649 keeps the skill, script, settings, eval and catalog items; this issue is blocked until that work merges.

Decision (owner, 2026-11-16)

Artifact retention stays at this repository's setting, which is also the most a public repository allows (gh api repos/melodic-software/claude-code-plugins/actions/permissions/artifact-and-log-retention returns days: 90, maximum_allowed_days: 90). No retention setting changes and no paid storage is requested. A pass dispatched more than 90 days after the previous one finds no baseline and reports every finding NEW.

Done when

  • A new workflow whose only trigger is manual dispatch by a listed human runs /testing:run-e2e map in the pull-request capture's sandbox, launch broker and key proxy on the pr-verify API key, and stops on the lanes kill switch.
  • The whole-map workflow takes features, max_entries (default 40) and baseline_run inputs and prints the entry count and the token cap before any model turn.
  • The whole-map workflow uploads findings.json at the repository's retention (90 days), registered with the workflow as its reader in the artifact retention registry, uploads the evidence with a short retention, and downloads the previous successful run's findings.json as the baseline; with no previous artifact it runs with no baseline.
  • The whole-map workflow drives sessions on the capture's configured driven model.

Owner step

Unchanged from #15649: the owner dispatches two measured passes and reviews them.

Activity

  1. added
    host-runNeeds an unattended run on a host; the host runner takes it, not a person or the issue lane.
    on Nov 16, 2026
  2. added
    needs-triageNot yet classified. Floor until a type and one priority tier are set.
    on Nov 16, 2026
  3. github-actions commented on Nov 16, 2026

    @github-actions
    Contributor

    Size: Medium
    Complexity: Medium
    Uncertainty: Knowable
    Flags: none
    Blocked by: #15649

    Understood: add a new workflow, triggered only by manual dispatch from a listed human, that runs /testing:run-e2e map in the pull-request capture's sandbox. It takes features, max_entries and baseline_run inputs, prints the entry count and token cap before any model turn, and stops on the lanes kill switch. It uploads findings.json at 90-day retention and the evidence at a short retention, and it uses the previous successful run's findings.json as the baseline. The issue states that it waits on the skill, script and settings work in #15649, and I took that as the blocker.

    The retention decision is in the issue body, not in a recorded decision comment. It asks for no retention change and no paid storage, so I found no charge to flag. The owner's two measured passes are host work, which this run can't do. Nothing in the issue looks like a duplicate, and no merged pull request already covers it.

    Automated triage. A maintainer reviews it.

  4. removed
    needs-triageNot yet classified. Floor until a type and one priority tier are set.
    on Nov 16, 2026
  5. melodic-automation-lanes commented on Nov 16, 2026

    @melodic-automation-lanes
    Contributor

    Not admitted to the issue lane: its done-when asks for what an unattended lane run cannot perform: item 1 (host-data: "A new workflow whose only trigger is manual dispatch by a listed human runs /..."); recommended split: keep the other items here as the admissible issue and move these to a child issue for the host queue (host-run); its title or stated work names a charge (retention), which needs an owner decision; it carries host-run. Admission takes the issue once each of those is met, when intake triages it again (.github/actions/resolve-config/admission-rules.json).

    Reason: cost

  6. melodic-automation-lanes commented on Nov 16, 2026

    @melodic-automation-lanes
    Contributor

    The latest triage verdict holds nothing: admission moved the issue to needs-owner-decision; its comment says why.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    host-runNeeds an unattended run on a host; the host runner takes it, not a person or the issue lane.

    Fields

    Work status

    needs-owner-decision

    Priority

    medium

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions