Skip to content

Say in the property guidance that it covers every target and that fuzzing follows it #15825

Open

Description

@kyle-sexton

Parent

Refs #15655 and #15649 (the map decisions on the talk's fuzzing and release pass, where the owner decided this).

Depends on

Outcome

/testing:write's property guidance says plainly that it covers every target and that fuzzing is the same mechanism, so nobody builds or looks for a separate fuzzing or UI-property skill.

Demo: ask /testing:write for a property test of a running UI, or for a fuzz test of a parser; it follows the one property guidance, finds the consumer's own tool, and names none itself.

Source: the talk "All Code Is Slop (Except Mine)" (Lauren Tan), slide 28 and the transcript at 1:12:39: "property-based testing as well for UIs". The tool she shows describes itself as "property-based testing, or fuzzing if you will".

Decisions

  • Owner, 2026-11-12: property testing is one thing with one home and is not split by target ("I think we just consolidate property testing and then split it based on the target"); flood-style fuzzing belongs with it.
  • A fuzz test is generated input with the weak rule "does not crash"; a property test is generated input with a stated rule. Same mechanism, same guidance.
  • Generated user actions against a running UI are a generated call sequence on a stateful thing, which the guidance already covers.
  • No tool is named or shipped; the tool comes from the consumer's repository.
  • This ships separately from the pilot cut so that cut's before-and-after result is not confounded by added text.

Key interfaces

None.

Done when

  • The property guidance states, in one or two sentences, that it applies to every target (a running UI included) and that a fuzz test follows it; it names no tool.
  • One new eval case, frozen first, covers the sentences; the skill's existing write cases are non-inferior on a two-version compare (opus and sonnet aliases, three runs per side, --case filter, a --max-cost-usd ceiling, no new batch after a usage-limit error), run in the one local eval-runner session; the pull request carries do-not-merge until the record lands.
  • The pull request body names this repository's dogfooding step.

Out of scope

  • Any per-language or per-tool page.
  • The release pass.

Activity

  1. added
    agent-readyFully specified and briefed; eligible for autonomous pickup from the frontier.
    on Nov 13, 2026
  2. added theissue type on Nov 13, 2026
  3. removed
    agent-readyFully specified and briefed; eligible for autonomous pickup from the frontier.
    on Nov 13, 2026
  4. github-actions commented on Nov 13, 2026

    @github-actions
    Contributor

    Size: Small
    Complexity: Low
    Uncertainty: Known
    Flags: none
    Blocked by: none

    Understood: add one or two sentences to the /testing:write property guidance saying it applies to every target (a running UI included) and that a fuzz test follows it. The sentences name no tool. The change also needs one new frozen eval case and a non-inferiority compare on the existing write cases, and the pull request body names the dogfooding step.

    Automated triage. A maintainer reviews it.

  5. melodic-automation-lanes commented on Nov 13, 2026

    @melodic-automation-lanes
    Contributor

    Not admitted to the issue lane: merged pull request #16464 names it with Refs: after its newest decision or refinement comment, so its work may already be done: close the issue if it is, or post a comment whose first line is **Decision recorded.** saying what is left, which sends it back through intake. Admission takes the issue once each of those is met, when intake triages it again (.github/actions/resolve-config/admission-rules.json).

  6. melodic-automation-lanes commented on Nov 15, 2026

    @melodic-automation-lanes
    Contributor

    {"v":1,"actor_id":337595936,"title_sha256":"a522e6a34f2e287f8ac73c60603a525bca8dfaa52aa4c35d400932f270c43b36","body_sha256":"2e81b8949b5679bf96df16984207b862439f7aad32d4f2058e3bb8b2fc874ea9"}

  7. github-actions commented on Nov 15, 2026

    @github-actions
    Contributor
    No description provided.
  8. melodic-automation-lanes commented on Nov 15, 2026

    @melodic-automation-lanes
    Contributor

    Pull request #16464 merged with Refs:, so this issue stays open and nothing starts the lane on it again. work-unblock removed status: ready and agent-ready, so the hourly triage sweep sends it back to intake triage. Admission then parks it at status: confirmed for a check of what that pull request left, until a **Decision recorded.** comment says what is left or someone closes it.

  9. github-actions commented on Nov 16, 2026

    @github-actions
    Contributor

    The triage sweep dispatched intake triage for this issue (dispatch 1; 2 runs that reach intake, or 6 dispatches, hand it to a maintainer).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    Fields

    Work status

    confirmed

    Priority

    low

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions