docs: correct the portable hint-table coverage claims for this fork - #17
Merged
Merged
Conversation
The hint tables' five measured CI runs are upstream's, and they cover the lane sizes upstream held when they ran. This fork's serial lane carries additional fork-only members no upstream run measured, so state that they pack on the PORTABLE_SERIAL_DEFAULT_WEIGHT_MS default until the fork refreshes its own hints, and point at bin/fm-test-run.sh --check-coverage for the live lane size and unmeasured share instead of a count copied here. Generalize the refresh command's -R owner placeholder to match.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Intent
Land a docs-only correction on keenvc/firstmate: docs/fm-test-portable-shards.md claimed the measured hint tables cover exactly all 24 parallel and 201 serial members, but those five measured CI runs are upstream's and this fork's serial lane carries additional fork-only members no upstream run measured - state that fork-only members pack on the PORTABLE_SERIAL_DEFAULT_WEIGHT_MS default until the fork refreshes its own hints from its own green CI runs, point at bin/fm-test-run.sh --check-coverage for the live lane size and unmeasured share, and generalize the refresh command's -R owner placeholder. This preserves the two accurate refinements from the superseded sync branch (fm/fm-upstream-sync-2026-09-30-2 / PR #15, closed as superseded because its code content already landed via #16's squash). Docs-only change; no live surface. Known infrastructure quirk: the push-target binding opens the PR against upstream kunchenguid/firstmate where contributor-approval action_required gates block CI - infrastructure, not code, as established in prior runs.
What Changed
docs/fm-test-portable-shards.mdso the five 2026-09-30 CI runs are attributed to upstream and described as covering upstream's 24 parallel and 201 serial members, instead of claiming to cover this fork's serial lane.PORTABLE_SERIAL_DEFAULT_WEIGHT_MSdefault until the fork refreshes its own hints from its own green CI runs, points readers atbin/fm-test-run.sh --check-coveragefor the live lane size and unmeasured share, and generalizes the refresh command's-Rowner placeholder to<owner>/firstmate.ci.ymlwith PyYAML when ruby is unavailable.Risk Assessment
✅ Low: The authored change is a docs-only, source-verified correction whose claims match bin/fm-test-run.sh behavior and --check-coverage output, with no live surface or executable path altered.
Testing
I drove the live product the changed document points at, not just the prose.
bin/fm-test-run.sh --check-coverageexits 0 and reports the live portable-serial lane size (serial=211) and unmeasured share (serial_unhinted=10) the doc tells readers to use; the runner's measured hint table holds exactly 201 serial members, and the 10 live members without a hint are exactly the fork-only test files added on this fork versus upstream base 90cd351, so the doc's corrected claim that fork-only members pack on PORTABLE_SERIAL_DEFAULT_WEIGHT_MS is true. The targeted runner contract suitetests/fm-test-run.test.shpassed end-to-end against the real runner. The one remaining item, the prose placeholder generalization for the refresh command's -R owner, has no runtime surface to drive and is therefore reported as untested rather than passed. Evidence files were written under the run evidence directory; the worktree is clean.bin/fm-test-run.sh --check-coverageand sees the live portable-serial lane size and unmeasured share--list --lane portable-serial= 211; measured hint table = 201; comm of the two = the 10 fork-only test files (also confirmed absent from base 90cd351); see docs-correction-live-validation.mdbash tests/fm-test-run.test.shexit 0; log artifact fm-test-run-contract-20261002T053913Z.log-R <owner>/firstmateplaceholder and gh-axi accepts that flag formEvidence: Live coverage summary from the runner the doc points at
Evidence: Docs correction live validation (round 2)
Source: Docs correction live validation (round 2)
Evidence: Runner contract suite log (tests/fm-test-run.test.sh, exit 0)
Source: Runner contract suite log (tests/fm-test-run.test.sh, exit 0)
Evidence: Doc diff under test (docs/fm-test-portable-shards.md @ 916ad55)
Source: Doc diff under test (docs/fm-test-portable-shards.md @ 916ad553)
Pipeline
Updates from git push no-mistakes
✅ **intent** - passed
✅ No issues found.
.agents/skills/afk/SKILL.md- branch carries 48 commit(s) that exist on your local main branch but were never pushed to origin/main; these may be unintended bundled work (proposed PR changes 396 file(s)):Confirm these commits belong in this PR before approving, or manually separate the intended work onto origin/main before gating.
🔧 No changes applied.
1 warning still open:
.agents/skills/afk/SKILL.md- branch carries 48 commit(s) that exist on your local main branch but were never pushed to origin/main; these may be unintended bundled work (proposed PR changes 396 file(s)):Confirm these commits belong in this PR before approving, or manually separate the intended work onto origin/main before gating.
no changes applied: bundled local-default commits require manual separation or explicit approval; the rebase conflict resolver cannot safely select commits to discard.
✅ **Review** - passed
✅ No issues found.
🔧 **Test** - 1 issue found → auto-fixed ✅
-R <owner>/firstmateinstead of the hardcoded upstream owner: Docs-only change; the edit is prose and code-fence text in a Markdown document, which has no runtime surface to drive. Verified by reading the owned document text.-R <owner>/firstmateinstead of the hardcoded upstream ownergit show 916ad553 --stat— confirms the change touches only docs/fm-test-portable-shards.md (1 file, 3 insertions, 2 deletions)bin/fm-test-run.sh --check-coverage(live, exit 0) — reportstotal=251 parallel=24 parallel_unhinted=0 serial=211 serial_shards=9 serial_unhinted=10, proving the doc's--check-coveragepointer yields the live lane size (serial=) and unmeasured share (serial_unhinted=)bin/fm-test-run.sh --list --lane portable-serial | wc -l(211) vsportable_serial_weight_hintsentries (201) — the 10-member delta equals the reportedserial_unhinted=10comm -23of the live portable-serial members against the embedded hint paths — lists the 10 unmeasured members as fork-only tests (fm-claim, fm-cline-harness, fm-cline-signals-live-e2e, fm-hold-reverify, fm-openhands-harness, fm-openhands-signals-live-e2e, fm-provider-lane-cap, fm-quota-wall-live-e2e, fm-watcher-continuity, fm-backend-herdr-probe-timeout)grep -n 'PORTABLE_SERIAL_DEFAULT_WEIGHT_MS' bin/fm-test-run.sh— confirms the named default exists (=45000)grep -n '\-R ' docs/fm-test-portable-shards.md— confirms the refresh command now uses-R <owner>/firstmateand no longer hardcodes the upstream owner in that command; remaining kunchenguid/firstmate strings are historical CI run and PR linksgh pr diff 15 -R keenvc/firstmate— confirms the superseded sync branch carried the same two refinements (coverage-claim correction and -R owner generalization) that this change restoresgit status --porcelain— clean worktree after validation🔧 Fix applied.
✅ Re-checked - no issues remain.
bin/fm-test-run.sh --check-coverageand sees the live portable-serial lane size and unmeasured share--list --lane portable-serial= 211; measured hint table = 201; comm of the two = the 10 fork-only test files (also confirmed absent from base 90cd351); see docs-correction-live-validation.mdbash tests/fm-test-run.test.shexit 0; log artifact fm-test-run-contract-20261002T053913Z.log-R <owner>/firstmateplaceholder and gh-axi accepts that flag formbin/fm-test-run.sh --check-coverage->FM_TEST_COVERAGE ok total=251 parallel=24 ... serial=211 ... serial_unhinted=10 serial_max_ms=1074843 serial_budget_ms=1200000 herdr=16, exit 0bin/fm-test-run.sh --list --lane portable-serial | wc -l-> 211; measured hint-table members -> 201;comm -23of the two -> exactly the 10 fork-only test files (fm-claim, fm-cline-harness, fm-cline-signals-live-e2e, fm-hold-reverify, fm-openhands-harness, fm-openhands-signals-live-e2e, fm-provider-lane-cap, fm-quota-wall-live-e2e, fm-watcher-continuity, fm-backend-herdr-probe-timeout)git ls-tree -r --name-only 90cd351a -- tests/vsls tests/*.test.sh-> all 10 unhinted members are fork-only additions absent from the upstream base 90cd351abash tests/fm-test-run.test.sh(targeted contract suite driving the real runner) -> exit 0, including 'coverage guard bounds the unmeasured share and serial packing within twenty minutes' and the pyyaml-based Herdr timeout parseRead the corrected paragraph indocs/fm-test-portable-shards.mdandgit show 916ad553 -- docs/fm-test-portable-shards.mdto confirm the three intent items (coverage-claim correction, --check-coverage pointer, generalized owner placeholder)docs/fm-test-portable-shards.md:87- The superseded branch fm/fm-upstream-sync-2026-09-30-2 (commit b9ebe4c) carried a third doc line after the refresh command: 'Name the repository whose lane you are refreshing: an upstream run never executes a fork-only member, so it cannot supply that member's hint.' This change omits it. That looks deliberate: the hand-written commit message for 916ad55 enumerates exactly two carried refinements (the coverage-claim correction and the -R owner generalization), and line 16 plus the <owner> placeholder already state that the fork must refresh from its own green runs, so the line would restate an existing fact. Recorded as the one delta from the superseded branch's doc change; no edit made.✅ **Lint** - passed
✅ No issues found.
✅ **Push** - passed
✅ No issues found.