Skip to content

Run a whole-map live verification pass on demand and report what is new since the last pass - part 1 of 6 #16566

Description

@melodic-automation-lanes

Part 1 of 6 of #15649, split at intake (owner Q93). The done-when below is copied from #15649 as written.

Done when

  • The testing plugin's findings compare script takes two findings.json files and labels each finding id NEW, PRE-EXISTING, GONE or not checked, with no model call.
  • The testing plugin's findings compare script never labels an id GONE when its entry was not driven in the newer pass, and labels every finding NEW when no baseline is given.
  • The testing plugin's findings compare script has a test file covering each label and the not-checked rule, run by CI.
  • The testing plugin's findings.json schema holds per finding the id, feature, part, kind, where, entry points, severity, stated or unstated label and evidence paths, and per pass the build and each entry's outcome, wall time and tokens.

Activity

  1. added
    needs-triageNot yet classified. Floor until a type and one priority tier are set.
    on Nov 16, 2026
  2. github-actions commented on Nov 16, 2026

    @github-actions
    Contributor

    Size: Medium
    Complexity: Low
    Uncertainty: Known
    Flags: none
    Blocked by: none

    Understood: this is part 1 of 6 of #15649. It asks for a no-model findings compare script in the testing plugin. The script labels each finding id NEW, PRE-EXISTING, GONE or not checked. It never labels an id GONE when its entry was not driven in the newer pass, and it labels everything NEW when no baseline is given. The issue also asks for a CI-run test file covering each label and the not-checked rule, and for a findings.json schema with the listed per-finding and per-pass fields. The done-when items are exact, so nothing is missing. This is a split child of #15649, not a duplicate. I found no merged change that already covers it.

    Automated triage. A maintainer reviews it.

  3. removed
    needs-triageNot yet classified. Floor until a type and one priority tier are set.
    on Nov 16, 2026
  4. melodic-automation-lanes commented on Nov 16, 2026

    @melodic-automation-lanes
    ContributorAuthor

    {"v":1,"actor_id":337595936,"title_sha256":"b2a92b15872509946b548697e2ee34c79166913fda2ba06681ea0c5c9722a54a","body_sha256":"3a3fd6f1bd88a83bc6647b6605ce040f8e061703f76d2956ea8e4e49e8cad619"}

  5. github-actions commented on Nov 16, 2026

    @github-actions
    Contributor
    No description provided.
  6. github-actions commented on Nov 16, 2026

    @github-actions
    Contributor
    No description provided.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    Fields

    Work status

    agent-ready

    Priority

    medium

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions