Repository navigation
docs(release-criteria): W5 counts a held-out pair by its first measurement (#408) - #1294
Merged
Merged
Conversation
…ement; a pair that fails it stays a quotable regression check, and the six 2018 to 2021 pairs are used up (#408)
DrunkOnJava
enabled auto-merge (squash)
November 3, 2026 17:31
DrunkOnJava
added a commit
that referenced
this pull request
Nov 3, 2026
…final 1.0 check (#408) (#1296) ## Summary A fresh held-out pair offered for W5 (#408) is pinned sealed: no Measure run fetches it until the final 1.0 check. That check is made once, when W4, W6 and W7 pass on the reference models. ## Motivation Since #1294, W5 counts a pair only if its first measurement meets W4, W6 and W7. No model meets W6 (some bodies are still boxes) or W7 (some properties are still missing). So any pair measured now fails at its first measurement and is used up. `-f holdout=true` runs measure every pair on the held-out list, and they run often. The pairs asked of contributors on 2026-10-29 would be used up on their first such run. ## Changes - `docs/release-1.0-criteria.md`, W5: the sealing rule. - `docs/reference-models.md`, "Adding a model": the same for whoever offers a pair. No pair is sealed yet. The pin list and the Measure input for the final check come with the first sealed pair. ## Testing Docs only.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
W5 now states how a held-out pair counts, as Griffin decided on 2026-11-03 ("first measurement decides").
A pair counts for W5 only if its first measurement, made before any fix is made for it, meets W4, W6 and W7. A pair whose first measurement does not is used up for W5. It stays in Measure as a held-out regression check: counts only, never developed on, and its counts may be quoted in reports.
Motivation
W5 counted a pair only if it was "used in no RE report". RE-313, RE-314 and RE-315 quote held-out counts lines, and RE-315's extras on Vitoria's
ALD26 Projectpoint at #1286's next fix. Read strictly,ALD26 Projecthad already failed at its first measurement (0 of 1,520 elements), so the old wording left it unclear what a failed pair is for.Changes
docs/release-1.0-criteria.md, the W5 row:So they are used up for W5, and every release needs fresh pairs.
Testing
Docs only. Every figure is a count line of run 41658225355: the class-set tables and the GlobalId line for Frigento's 22.