Skip to content

docs: record what 2026-08-02 measured (RULES consumer-verify + declarations-vs-instances + output-format class, CLAUDE.md CI/byte facts, #273 tab discard, clawgate check-chart) - #287

Merged
ZacxDev merged 2 commits into
mainfrom
docs/measured-2026-08-02
Aug 2, 2026
Merged

docs: record what 2026-08-02 measured (RULES consumer-verify + declarations-vs-instances + output-format class, CLAUDE.md CI/byte facts, #273 tab discard, clawgate check-chart)#287
ZacxDev merged 2 commits into
mainfrom
docs/measured-2026-08-02

Conversation

@ZacxDev

@ZacxDev ZacxDev commented Aug 2, 2026

Copy link
Copy Markdown
Contributor

Docs-only. Records what a long 2026-08-02 session measured, into the four places that will be re-read rather than into a handoff doc. No source file is touched.

Every figure below was verified against the current tree, the closed issue, or live state before being written. Where I could not verify something, it is called out in Not verified at the bottom.


1. claude/RULES.md — three additions

(a) Verify the CONSUMER is running your artifact, not that the deploy reported success.Verification Honesty, immediately after "Deployed ≠ verified".

ship.sh reported ✅ VERIFIED — on branch main at origin/main + switched on the workbench while the browser-bridge systemd --user unit was crash-looping on OSError: [Errno 98] Address already in use. An orphaned process from the previous day (started Aug 1 16:18, in no systemd cgroup) held 127.0.0.1:8788 and was serving the OLD server.py. Every "deployed" claim about that host would have been measured against the orphan; the converge check verified branch + switch and structurally could not see that the service never started.

The recipe in the bullet was executed live before being written, not composed from memory:

ss -lptn 'sport = :8788'            → users:(("python3",pid=2202139,fd=3))
/proc/2202139/cgroup                → …/app.slice/browser-bridge.service
systemctl --user show browser-bridge -p MainPID -p SubState
                                    → MainPID=2202139  SubState=running

Cross-references the existing "resolve PIDs before killing, never let a -f pattern reach pkill" rule rather than restating it.

(b) A count of DECLARATIONS is not a count of INSTANCES.A Green Test Suite Is a Claim, after the "COUNT the tests" bullet.

A grep of skipif decorators found "2 node-related skips", reported as 2 skipped tests and used to size the work. The two decorators gated 123 tests (initiatives: 660 passed / 123 skipped in the sandbox vs 783 passed / 0 skipped with node). Re-confirmed in this run: scripts/initiatives/tests collected 783, skipped 0.

(c) Sharpened the harness bullet from a MANIFESTATION to a CLASS.

The file already listed "diff defaulting to unified output, so a byte-identical control passed" as one of nine harness failures. That rule was READ this session and the trap was hit anyway in a different shape, so the generalisation is now its own bullet, explicitly labelled as the class behind that item:

when you PARSE a tool's output, its format is a dependency you did not pin — cross-check against a second tool that fails differently (cmp vs diff, exit status vs parsed summary), and treat "no matches" as "possibly the wrong pattern" until a positive control shows the pattern can match.

Three 2026-08-02 instances recorded: diff's unified output making ^>/^< greps report "0 lines differ" for files differing by 1,445 bytes (a false CLEAN, where the recorded case was a false PASS — cmp settled it); node 24's TAP/spec reporter change emptying a ^# (tests|pass|fail) grep, read as "no output" rather than "wrong pattern"; and rc=$? after a pipeline reading echo's status.

I did not add a near-duplicate of the existing harness bullet — the sharpening points back at it.

2. CLAUDE.md — stale facts in the browser-bridge bullet

3. scripts/browser-bridge/reference/tabs-instances.md — the #273 measurement

🔴 scripts/browser-bridge/SKILL.md is UNCHANGED at 11,845 B — the content went into reference/, not the byte-capped core. A reader does not get anything wrong without it (the core does not discuss discards at all), so no eviction was needed.

New ### A discard assigns a NEW tabId and releases ownership (MEASURED, #273) subsection under the tab-ownership list. Forced discard via brave://discards → Urgent Discard → reactivate: tab 484065264484065273, ownedTabIdNone, emulate on the new id refused as not-owned. Consequence: the stale-documentEmulation hazard #273 was filed against is not reachable on this Chromium, because the map is keyed by tabId and the id does not survive.

Scope is stated as the issue states it, not stronger: one build, one profile, one discard mechanism; auto-discard under real memory pressure not exercised; onReplaced firing inferred, not observed. The load-bearing caveat is kept prominent — the safety is a property of the browser, not of the bridge; if a future Chromium preserves the id the defect returns silently.

4. claude/skills/clawgate/SKILL.md

  • Live version 0.7.82 (embedded kubeclaw chart 0.7.1). The VER= example moves to the next number and the "last known shipped" note becomes 0.7.82 — "derive the number from the LIVE pin, never from this file" stays primary, so no new stale-by-construction number is created. The "vendored 0.7.0 vs kubeclaw 0.7.1 → re-sync PENDING" note is now resolved.
  • New 🔴 bullet: make check-chart is only meaningful if ~/workspace/kubeclaw is current. check-chart depends on sync-chart, which rsyncs from that local clone; it sat at chart 0.3.14 against a vendored 0.7.1, so running check-chart would have clobbered the deployed chart and then reported a false failure — drift it had just created. Fix is git -C ~/workspace/kubeclaw fetch origin && merge --ff-only origin/trunk first. Verified live: that clone is now at 0.7.1 on trunk (fbd13aa), i.e. the fast-forward really happened.

🔴 Pre-existing RED gate found on origin/main — not caused by this PR

The task's stated baseline was 2920 collected / 2918 passed / 2 skipped. That baseline is from 81e2d76; origin/main is 4 commits ahead of it, and on current origin/main the pytest gate is RED:

FAIL  scripts/claude-hooks/tests/test_guard_core.py (missing directory)
TOTAL collected=3385  passed=3384  skipped=1  failed=0  (floor: 2850)
RESULT: FAIL

Cause: run-tests.sh's HERMETIC_DIRS deliberately names a file

# A FILE, not a dir, and deliberately so: scripts/claude-hooks/tests/ also
# holds hand-rolled scripts that call main() at import and sys.exit() …
scripts/claude-hooks/tests/test_guard_core.py

— but run_pytest() guards with if [ ! -d "$d" ], which rejects a regular file and marks it FAIL (missing directory).

Attribution: the entry is absent at 81e2d76 and was added by e21985a (#276). Impact: test_guard_core.py holds 913 tests (measured: 913 passed in 1.38s when run directly) that the gate is not running.

My tree is docs-only (git status shows exactly the four .md files), so this cannot be my doing. I have deliberately not fixed it — that is a source change outside this PR's scope and wants its own regression test. Flagging rather than merging through it, per "a permanently-red gate is worse than no gate."

This is also a live instance of both rules this PR adds: one line in a list (a declaration) covers 913 instances, and my own … ; echo "EXIT=$?" wrapper reported exit 0 for a run whose summary said RESULT: FAIL.

Suite counts (counted, not exit-code-read)

suite result
node — node --test --test-reporter=tap scripts/browser-bridge/tests/*.test.mjs # tests 468 / # pass 468 / # fail 0 / # skipped 0 ✅ matches the 468 baseline
pytest hermetic set collected=3385 passed=3384 skipped=1 failed=0, but RESULT: FAIL on the pre-existing test_guard_core.py classification bug above
scripts/initiatives/tests 783 passed, 0 skipped — confirms the 123-skip claim written into RULES.md
scripts/browser-bridge/tests 436 passed, 0 skipped

nix flake check was started; its result is reported in a follow-up comment rather than asserted here, since the pytest check it runs is already red on main for the reason above.

Not verified

🤖 Generated with Claude Code

ZacxDev and others added 2 commits August 2, 2026 17:31
… clawgate)

RULES.md — three additions, all from measured ground cases:

* Verify the CONSUMER is running your artifact, not that the deploy
  reported success. `ship.sh` reported "VERIFIED — on branch main at
  origin/main + switched" while the browser-bridge unit was crash-looping
  on `OSError: [Errno 98] Address already in use`; an orphaned process
  from the previous day (Aug 1 16:18, in NO systemd cgroup) held
  127.0.0.1:8788 and served the OLD server.py. The converge check
  verified branch + switch and structurally could not see that the
  service never started. Recipe (`ss -lptn` -> `/proc/<pid>/cgroup` ->
  `systemctl show -p MainPID`) was executed live before being written.

* A count of DECLARATIONS is not a count of INSTANCES. A grep of
  `skipif` decorators found "2 node-related skips"; the two decorators
  gated 123 tests (initiatives: 660 passed/123 skipped sandboxed vs
  783 passed/0 skipped with node). A 60x sizing error.

* Sharpen the harness bullet from a MANIFESTATION to a CLASS: when you
  parse a tool's OUTPUT, its format is an unpinned dependency, and "no
  matches" means "possibly the wrong pattern". The file already named
  `diff`'s unified default; that rule was READ this session and the trap
  was hit anyway in three new shapes (a false CLEAN over a 1,445-byte
  difference that only `cmp` caught; node 24's reporter change emptying
  a `^# (tests|pass|fail)` grep; `rc=$?` reading `echo`'s status).

CLAUDE.md — correct stale facts in the browser-bridge bullet:

* Drop the hand-pinned "281 B free today". test_skill_size.py owns
  MAX_BYTES/MIN_HEADROOM_BYTES (the floor was raised to 250 in #275);
  point at it instead of re-pinning a figure that rots.
* Record that CI now gates BOTH suites — nodetests (#280) and the
  pytest gate's silent-coverage-collapse guards (#284) — and the
  headline outcome, skips 125 -> 2.
* Extension 0.7.1; `nav`/`open` accept `--wake[=MS]`.

reference/tabs-instances.md — record the #273 measurement: a forced
discard assigns a NEW tabId and releases ownership (484065264 ->
484065273, ownedTabId None), so the stale-documentEmulation hazard is
not reachable on this Chromium. Scope stated honestly (one build, one
profile, one mechanism; auto-discard not exercised; onReplaced inferred,
not observed) and the load-bearing caveat kept: the safety is a property
of the BROWSER, not of the bridge. Placed in reference/, not the
byte-capped core — SKILL.md is unchanged at 11,845 B.

clawgate/SKILL.md — live version 0.7.82 (embedded kubeclaw chart 0.7.1,
so the pending re-sync note is resolved); "derive from the LIVE pin"
stays primary. Adds the `make check-chart` hazard: it depends on
`sync-chart`, which rsyncs from ~/workspace/kubeclaw — that clone sat at
0.3.14 against a vendored 0.7.1, so running it would have clobbered the
deployed chart and reported a false failure. Fetch + `merge --ff-only`
that clone first.

Docs-only. No source file touched.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…s-vs-instances instances

Both measured while unbreaking the pytest gate (#289).

TWO TIERS. 'Gate on the merged tree' extended to environments. The same
suite runs in the nix sandbox and on a dev host, and each environment
silently decides which tests execute — so a defect can be permanently
unobservable in the tier you happen to read. #276 shipped THREE
regressions that masked each other:

  (1) a FILE in run-tests.sh's target list rejected by `[ ! -d ]`
      -> gate red, 913 tests never ran
  (2) SECRET_PATTERNS moved to guard_core.py -> the drift test parses []
      and FAILS on a host with the hook deployed, SKIPS in the sandbox
  (3) 10 nix-instantiate tests pytest.fail() without the binary
      -> FAIL in the sandbox, pass on every dev host

(2) and (3) are exact complements; both hid behind (1)'s red.

DECLARATIONS VS INSTANCES gets its third instance in one session:
2 skipif decorators -> 123 tests; 1 list entry -> 913 tests;
1 nix_eval() helper -> 10 parametrized tests. Three is a pattern.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@ZacxDev
ZacxDev force-pushed the docs/measured-2026-08-02 branch from c09c98c to 537723b Compare August 2, 2026 22:32
@ZacxDev
ZacxDev merged commit c9cb47a into main Aug 2, 2026
@ZacxDev
ZacxDev deleted the docs/measured-2026-08-02 branch August 2, 2026 22:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant