docs: record what 2026-08-02 measured (RULES consumer-verify + declarations-vs-instances + output-format class, CLAUDE.md CI/byte facts, #273 tab discard, clawgate check-chart) - #287
Merged
Conversation
… clawgate) RULES.md — three additions, all from measured ground cases: * Verify the CONSUMER is running your artifact, not that the deploy reported success. `ship.sh` reported "VERIFIED — on branch main at origin/main + switched" while the browser-bridge unit was crash-looping on `OSError: [Errno 98] Address already in use`; an orphaned process from the previous day (Aug 1 16:18, in NO systemd cgroup) held 127.0.0.1:8788 and served the OLD server.py. The converge check verified branch + switch and structurally could not see that the service never started. Recipe (`ss -lptn` -> `/proc/<pid>/cgroup` -> `systemctl show -p MainPID`) was executed live before being written. * A count of DECLARATIONS is not a count of INSTANCES. A grep of `skipif` decorators found "2 node-related skips"; the two decorators gated 123 tests (initiatives: 660 passed/123 skipped sandboxed vs 783 passed/0 skipped with node). A 60x sizing error. * Sharpen the harness bullet from a MANIFESTATION to a CLASS: when you parse a tool's OUTPUT, its format is an unpinned dependency, and "no matches" means "possibly the wrong pattern". The file already named `diff`'s unified default; that rule was READ this session and the trap was hit anyway in three new shapes (a false CLEAN over a 1,445-byte difference that only `cmp` caught; node 24's reporter change emptying a `^# (tests|pass|fail)` grep; `rc=$?` reading `echo`'s status). CLAUDE.md — correct stale facts in the browser-bridge bullet: * Drop the hand-pinned "281 B free today". test_skill_size.py owns MAX_BYTES/MIN_HEADROOM_BYTES (the floor was raised to 250 in #275); point at it instead of re-pinning a figure that rots. * Record that CI now gates BOTH suites — nodetests (#280) and the pytest gate's silent-coverage-collapse guards (#284) — and the headline outcome, skips 125 -> 2. * Extension 0.7.1; `nav`/`open` accept `--wake[=MS]`. reference/tabs-instances.md — record the #273 measurement: a forced discard assigns a NEW tabId and releases ownership (484065264 -> 484065273, ownedTabId None), so the stale-documentEmulation hazard is not reachable on this Chromium. Scope stated honestly (one build, one profile, one mechanism; auto-discard not exercised; onReplaced inferred, not observed) and the load-bearing caveat kept: the safety is a property of the BROWSER, not of the bridge. Placed in reference/, not the byte-capped core — SKILL.md is unchanged at 11,845 B. clawgate/SKILL.md — live version 0.7.82 (embedded kubeclaw chart 0.7.1, so the pending re-sync note is resolved); "derive from the LIVE pin" stays primary. Adds the `make check-chart` hazard: it depends on `sync-chart`, which rsyncs from ~/workspace/kubeclaw — that clone sat at 0.3.14 against a vendored 0.7.1, so running it would have clobbered the deployed chart and reported a false failure. Fetch + `merge --ff-only` that clone first. Docs-only. No source file touched. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…s-vs-instances instances Both measured while unbreaking the pytest gate (#289). TWO TIERS. 'Gate on the merged tree' extended to environments. The same suite runs in the nix sandbox and on a dev host, and each environment silently decides which tests execute — so a defect can be permanently unobservable in the tier you happen to read. #276 shipped THREE regressions that masked each other: (1) a FILE in run-tests.sh's target list rejected by `[ ! -d ]` -> gate red, 913 tests never ran (2) SECRET_PATTERNS moved to guard_core.py -> the drift test parses [] and FAILS on a host with the hook deployed, SKIPS in the sandbox (3) 10 nix-instantiate tests pytest.fail() without the binary -> FAIL in the sandbox, pass on every dev host (2) and (3) are exact complements; both hid behind (1)'s red. DECLARATIONS VS INSTANCES gets its third instance in one session: 2 skipif decorators -> 123 tests; 1 list entry -> 913 tests; 1 nix_eval() helper -> 10 parametrized tests. Three is a pattern. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
ZacxDev
force-pushed
the
docs/measured-2026-08-02
branch
from
August 2, 2026 22:32
c09c98c to
537723b
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Docs-only. Records what a long 2026-08-02 session measured, into the four places that will be re-read rather than into a handoff doc. No source file is touched.
Every figure below was verified against the current tree, the closed issue, or live state before being written. Where I could not verify something, it is called out in Not verified at the bottom.
1.
claude/RULES.md— three additions(a) Verify the CONSUMER is running your artifact, not that the deploy reported success. → Verification Honesty, immediately after "Deployed ≠ verified".
ship.shreported✅ VERIFIED — on branch main at origin/main + switchedon the workbench while thebrowser-bridgesystemd --userunit was crash-looping onOSError: [Errno 98] Address already in use. An orphaned process from the previous day (started Aug 1 16:18, in no systemd cgroup) held127.0.0.1:8788and was serving the OLDserver.py. Every "deployed" claim about that host would have been measured against the orphan; the converge check verified branch + switch and structurally could not see that the service never started.The recipe in the bullet was executed live before being written, not composed from memory:
Cross-references the existing "resolve PIDs before killing, never let a
-fpattern reachpkill" rule rather than restating it.(b) A count of DECLARATIONS is not a count of INSTANCES. → A Green Test Suite Is a Claim, after the "COUNT the tests" bullet.
A grep of
skipifdecorators found "2 node-related skips", reported as 2 skipped tests and used to size the work. The two decorators gated 123 tests (initiatives: 660 passed / 123 skipped in the sandbox vs 783 passed / 0 skipped withnode). Re-confirmed in this run:scripts/initiatives/testscollected 783, skipped 0.(c) Sharpened the harness bullet from a MANIFESTATION to a CLASS.
The file already listed "
diffdefaulting to unified output, so a byte-identical control passed" as one of nine harness failures. That rule was READ this session and the trap was hit anyway in a different shape, so the generalisation is now its own bullet, explicitly labelled as the class behind that item:Three 2026-08-02 instances recorded:
diff's unified output making^>/^<greps report "0 lines differ" for files differing by 1,445 bytes (a false CLEAN, where the recorded case was a false PASS —cmpsettled it); node 24's TAP/spec reporter change emptying a^# (tests|pass|fail)grep, read as "no output" rather than "wrong pattern"; andrc=$?after a pipeline readingecho's status.I did not add a near-duplicate of the existing harness bullet — the sharpening points back at it.
2.
CLAUDE.md— stale facts in the browser-bridge bulletMAX_BYTES12,288,MIN_HEADROOM_BYTES250 (raised from 100 in perf(browser-bridge): drophealthfrom the quick start; fix + tighten the SKILL.md size gate #275) → 443 B headroom, 193 B usable. Rather than re-pin a figure that rots, the bullet now namesMAX_BYTES/MIN_HEADROOM_BYTESand says to read them intests/test_skill_size.py— consistent with how the bullet already defers on the ceiling.checks.x86_64-linux.nodetests(ci: gate the browser-bridge node suite in the flake (460 tests, counted) #280) gates the.mjssuite (468 measured today; the runner asserts the floor) parsing TAP rather than reading an exit code, andchecks.x86_64-linux.pytests(test(gate): count skips in the pytest gate — tool precondition, pinned skip set, collected-test floors #284) fails loudly on silent coverage collapse via a required-tools precondition, a pinned skip set, and per-directory + global collected-test floors. Headline outcome recorded: skips 125 → 2.manifest.json);nav/openaccept--wake[=MS](verified in thebrowserCLI).3.
scripts/browser-bridge/reference/tabs-instances.md— the #273 measurement🔴
scripts/browser-bridge/SKILL.mdis UNCHANGED at 11,845 B — the content went intoreference/, not the byte-capped core. A reader does not get anything wrong without it (the core does not discuss discards at all), so no eviction was needed.New
### A discard assigns a NEW tabId and releases ownership (MEASURED, #273)subsection under the tab-ownership list. Forced discard viabrave://discards→ Urgent Discard → reactivate: tab484065264→484065273,ownedTabId→None,emulateon the new id refused as not-owned. Consequence: the stale-documentEmulationhazard #273 was filed against is not reachable on this Chromium, because the map is keyed by tabId and the id does not survive.Scope is stated as the issue states it, not stronger: one build, one profile, one discard mechanism; auto-discard under real memory pressure not exercised;
onReplacedfiring inferred, not observed. The load-bearing caveat is kept prominent — the safety is a property of the browser, not of the bridge; if a future Chromium preserves the id the defect returns silently.4.
claude/skills/clawgate/SKILL.mdVER=example moves to the next number and the "last known shipped" note becomes 0.7.82 — "derive the number from the LIVE pin, never from this file" stays primary, so no new stale-by-construction number is created. The "vendored 0.7.0 vs kubeclaw 0.7.1 → re-sync PENDING" note is now resolved.make check-chartis only meaningful if~/workspace/kubeclawis current.check-chartdepends onsync-chart, which rsyncs from that local clone; it sat at chart 0.3.14 against a vendored 0.7.1, so runningcheck-chartwould have clobbered the deployed chart and then reported a false failure — drift it had just created. Fix isgit -C ~/workspace/kubeclaw fetch origin && merge --ff-only origin/trunkfirst. Verified live: that clone is now at 0.7.1 ontrunk(fbd13aa), i.e. the fast-forward really happened.🔴 Pre-existing RED gate found on
origin/main— not caused by this PRThe task's stated baseline was
2920 collected / 2918 passed / 2 skipped. That baseline is from81e2d76;origin/mainis 4 commits ahead of it, and on currentorigin/mainthe pytest gate is RED:Cause:
run-tests.sh'sHERMETIC_DIRSdeliberately names a file —— but
run_pytest()guards withif [ ! -d "$d" ], which rejects a regular file and marks itFAIL (missing directory).Attribution: the entry is absent at
81e2d76and was added bye21985a(#276). Impact:test_guard_core.pyholds 913 tests (measured:913 passed in 1.38swhen run directly) that the gate is not running.My tree is docs-only (
git statusshows exactly the four.mdfiles), so this cannot be my doing. I have deliberately not fixed it — that is a source change outside this PR's scope and wants its own regression test. Flagging rather than merging through it, per "a permanently-red gate is worse than no gate."This is also a live instance of both rules this PR adds: one line in a list (a declaration) covers 913 instances, and my own
… ; echo "EXIT=$?"wrapper reported exit 0 for a run whose summary saidRESULT: FAIL.Suite counts (counted, not exit-code-read)
node --test --test-reporter=tap scripts/browser-bridge/tests/*.test.mjs# tests 468 / # pass 468 / # fail 0 / # skipped 0✅ matches the 468 baselinecollected=3385 passed=3384 skipped=1 failed=0, butRESULT: FAILon the pre-existingtest_guard_core.pyclassification bug abovescripts/initiatives/tests783 passed, 0 skipped— confirms the 123-skip claim written into RULES.mdscripts/browser-bridge/tests436 passed, 0 skippednix flake checkwas started; its result is reported in a follow-up comment rather than asserted here, since the pytest check it runs is already red onmainfor the reason above.Not verified
ship.sh/orphan incident are from the session that produced them; I verified the mechanism live (cgroup/MainPID recipe) and the current skip count (1 pinned skip in this run), but I did not re-run the pre-test(gate): count skips in the pytest gate — tool precondition, pinned skip set, collected-test floors #284 tree to reproduce 125.node 24's reporter change is recorded as reported. This host's PATH node is v26.5.0 and the flake sandbox pins v24.18.0; I did not bisect the reporter behaviour across those.onReplacedfiring on discard — inferred in browser-bridge: documentEmulation may survive a tab discard-reload and silence documentPredatesEmulation #273, and written into the doc as inferred.🤖 Generated with Claude Code