Skip to content

Let a verified runtime trace from an allowlisted monitor conclude PASS for RE-02.01 #553

Description

@mlieberman85

Feature 044 (#453) registers every reproducibility step type that decides from text or file-presence signals as FAIL-only. A PASS from those signals is evidence only.

repro_hermetic_build also verifies Witness attestations. A verified attestation that records no network access during the build is a cryptographic observation, not a text signal. Because it sits inside the same step type, it can no longer conclude PASS for RE-02.01 either.

Proposal:

  • Split Witness verification into its own step type with a {pass, fail} ceiling. It passes on a verified attestation with no recorded network access, fails on a verified attestation that records network access, and is ERROR when the attestation can't be verified.
  • Add it as an RE-02.01 step ahead of repro_hermetic_build.
  • Back it with corpus cases.

The signal-based checks in repro_hermetic_build stay FAIL-only.

Related: #453, #227. Paths change if #532 (rename to darnit-amber) lands first.

(Drafted with Claude Code.)

Activity

  1. changed the title [-]Let a verified Witness attestation conclude PASS for RE-02.01 (split it from repro_hermetic_build)[/-] [+]Let a verified runtime trace from an allowlisted monitor conclude PASS for RE-02.01[/+] on Oct 8, 2026
  2. mlieberman85 commented on Oct 8, 2026

    @mlieberman85
    ContributorAuthor

    #565 did the split, but the new repro_witness_attestation step is FAIL-only ({fail} ceiling), not {pass, fail}.

    Why not PASS: a verified empty network log doesn't prove there was no network access.

    • Under the in-toto parsing rules, an absent or null optional list counts the same as an empty one.
    • The runtime-trace v0.1 spec leaves what monitorLog.network records to monitor.type and its tracePolicy.
    • A monitor that doesn't trace sockets produces the same empty log.
    • The spec's own Tetragon example (with a connect policy) has no network field at all.

    Also in #565:

    • Verification is bound to the audited commit (GitHubWorkflowSHA).
    • Network events are read only from the exact runtime-trace predicate type.
    • A verified trace that records network access concludes RE-02.01 FAIL.

    I've retitled this issue to track what's left. To PASS RE-02.01 from a runtime trace, we need:

    • a built-in allowlist of monitor types, each with the trace policy that shows it records network access (for example, Tetragon with a connect policy);
    • corpus cases built from real attestations, not synthetic statements;
    • a widened ceiling, or a corpus-backed promotion, only for traces from allowlisted monitors.

    Contract: specs/044-false-pass-paths/contracts/witness-step-addendum-553.md.

    Drafted with Claude Code; reviewed and posted by me.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions