test(browser-bridge): anchor the agent surface to the real op inventory (contains 1 DELIBERATE failure)

[ZacxDev]

🔴 This PR contains ONE deliberately-failing test. The failure IS the finding.

✖ AGENT SURFACE PARTITION: every wire op is REACHABLE or a DECLARED exclusion
  UNDECLARED EXCLUSION(S): context, emulate, ping — these wire ops are unreachable
  by the autonomous agent (absent from OP_TO_SERVER, so not even
  BROWSER_AGENT_ALLOWED_OPS can re-enable them) and carry NO written rationale,
  unlike every other exclusion in that file.

Do not "fix" this by tuning the test, and do not close it by inventing a
rationale — the test asserts that the cited rationale exists verbatim in
browser_tool_impl.mjs, so a fabricated one fails with a different, equally
loud message (mutation M19 below proves it). Whether each of these three should
be reachable is an operator decision, not one a test may make. Two legitimate
resolutions:

• (a) the op should be reachable → add it to OP_TO_SERVER and to all four
  agent-facing sources; or
• (b) the exclusion is deliberate → write the rationale comment in
  browser_tool_impl.mjs and add a REVIEWED_AGENT_EXCLUSIONS entry citing it.

open, close, tabs and activate are absent from the agent surface with a
careful rationale paragraph each. context, ping and emulate are absent
without one. This PR makes CI able to tell those two states apart.

Why

Per the 2026-08-02 surface audit (claudedocs/browser-bridge-surface-audit-2026-08-02.md,
#274) — findings F9/V3, gaps G1/G2.

The four-source agent parity test at browser_tool.test.mjs:846 pinned
browser.js's typed enum, the agent-md capability table, the README contract and
ALLOWED_OPS_DEFAULT to each other and to nothing upstream. All four omit
context, ping and emulate identically, so the set is self-consistent, CI is
green, and the omission is invisible. That is structurally the same defect as
context being dead on main (present in protocol.js, absent from
server.py's ALLOWED_OPS) — one layer further out, at the surface the agent
actually touches.

test_server.py:3751 already does this right for the wire layer: it parses
protocol.js and asserts set-equality against server.py. Same approach applied
here at two more layers — an authoritative upstream source, parsed, not a
hand-maintained list.

What's in it

tests/browser_tool.test.mjs — +6 tests (T1)

test	what it pins
HARNESS SELF-CHECK: … plausible, non-empty set	the inventory parses to ≥18 ops and contains known literals
HARNESS SELF-CHECK: … FAILS LOUDLY on an unreadable source	the parser raises rather than returning {}
UPSTREAM ANCHOR: every agent-facing op name resolves to a REAL wire op	each of the four sources → OP_TO_SERVER → server.py ALLOWED_OPS
UPSTREAM ANCHOR: every OP_TO_SERVER target is a wire op or a declared non-wire endpoint	catches a retargeted alias / typo'd mapping
DECLARED EXCLUSIONS: each carries a written rationale that EXISTS in the source	reason ≥40 chars and a source snippet found verbatim in browser_tool_impl.mjs
AGENT SURFACE PARTITION	the failing one. reachable ∪ declared-excluded == the full wire inventory, exactly

The pre-existing four-source test keeps every assertion it had; added ahead of the
bare deepEqual is a per-source diff so the message names which source is
missing which op rather than dumping two sorted arrays.

tests/test_surface_parity.py — new, +12 tests (T2, T3)

Written as pytest on purpose: flake.nix's checks.pytests (the repo's real
pre-merge gate) runs scripts/run-tests.sh --set hermetic, which covers
scripts/browser-bridge/tests for test_*.py only. The .mjs suites are
documented/manual (README.md:1693) and are not in the flake gate — so T2/T3
land where they will actually block a merge. ⚠️ That also means the deliberate T1
failure above is not enforced by the flake gate today; flagging that rather than
silently working around it.

T2 — CLI SUBCOMMANDS ↔ op inventory. Not a plain set-equality: 24 CLI names
vs 18 wire ops + 1 server op. The asymmetry is modelled explicitly — every name is
classified into exactly one of CLI_WIRE_OPS / CLI_SERVER_OPS / CLI_ALIASES /
CLI_CLIENT_ONLY (the last requires a written reason), the classification is
asserted exhaustive and a partition, and it is anchored to the CLI's real
cmd_op <op> call sites so the table cannot claim a mapping the script does not
make. An unclassified new CLI name fails by name.

T3 — SKILL.md ops table ↔ SUBCOMMANDS, both directions, plus a
server.py → SKILL.md test closing the triangle. Every failure message names the
specific op and the direction of the mismatch.

Verification

Counted from reporter lines, never exit codes. Base ref origin/main @ ff0fe41.

suite	at base ref	at HEAD
node --test tests/*.test.mjs	tests 454 / pass 454 / fail 0	tests 460 / pass 459 / fail 1 (the finding)
pytest tests/	369 passed, 0 failed	381 passed, 0 failed

Mutation sweep — 25 mutations, all red, each verified red for its OWN assertion

Deliberately varied in construction, not just count: deletions, renames,
additions, alias retargeting, an unclassified name, a fabricated rationale, and
harness sabotage. Every mutation was applied to a pristine in-memory copy and
reverted; M0 is an identity negative control confirming the sweep reproduces
the exact baseline before any verdict is read.

#	mutation	result
M0	identity — no change (negative control)	node 70/1 baseline, py 12/0 — sweep is trustworthy
M1	drop wake from ALLOWED_OPS_DEFAULT only	✔ red, 4-source parity + partition
M2	drop wake from browser.js enum only	✔ red, names browser.js
M3	delete the activate rationale comment from the source	✔ red — DECLARED EXCLUSIONS, cited rationale absent
M4	drop frames from OP_TO_SERVER (silent surface shrink)	✔ red, UPSTREAM ANCHOR
M5	remove context from SUBCOMMANDS	✔ red — "STALE classification: context"
M6	remove the emulate row from SKILL.md's table	✔ red — undocumented subcommand
M7	remove screenshot from server.py ALLOWED_OPS	✔ red (10 tests incl. the harness spot-pin)
M8	rename screenshot→snap in agent-md only	✔ red — "agent op snap has NO OP_TO_SERVER entry"
M9	retarget the html alias → getHTML (case typo)	✔ red — "html → getHTML, NOT in ALLOWED_OPS"; py correctly unaffected
M10	rename CLI frames→iframes	✔ red, 3 tests
M11	change what the CLI dispatches (cmd_op tabs→tablist)	✔ red — "dispatches op(s) no classification entry claims: tablist"
M12	rename context→pagectx in SKILL.md's row only	✔ red, both directions
M13	add context to browser.js enum only	✔ red — one source ahead of the others
M14	add a new wire op sniff to server.py + protocol.js only	✔ red both suites — "wire op(s) with NO browser CLI subcommand: sniff". This is the G1 defect.
M15	add an unclassified CLI name sniff	✔ red — "UNCLASSIFIED CLI subcommand(s): sniff"
M16	document a phantom command teleport in SKILL.md	✔ red — "documents command(s) the CLI does not have"
M17	retarget the js alias to text	✔ red — "assert 'text' == 'eval'"
M18	silence M15 by parking sniff in CLI_CLIENT_ONLY with a stub reason	✔ red — the required-reason check refuses the stub
M19	declare context excluded citing a rationale that is NOT in the source	✔ red — "cited rationale is NOT present in browser_tool_impl.mjs". This is the anti-tuning property.
M20	declare text (a reachable op) as an exclusion	✔ red — "declared EXCLUDED but OP_TO_SERVER makes it reachable"
M21	break the SUBCOMMANDS regex (parser yields nothing)	✔ red — HARNESS BROKEN, not a vacuous green
M22	break the server.py ALLOWED_OPS regex	✔ red — HARNESS SELF-CHECK fires
M23	make the SKILL.md parser split on raw pipes (silently loses rows)	✔ red (loses emulate/key)

Right-reason check. For every mutation the full assertion text was captured
and inspected, confirming the test failed on its own assertion rather than an
earlier one throwing first. Two worth calling out:

• M14 changed the already-failing AGENT SURFACE PARTITION message from
  context, emulate, ping to context, emulate, ping, sniff — the added op is
  named, so the test discriminates rather than merely staying red.
• M19 made context disappear from the partition message (leaving
  emulate, ping) while raising a separate failure about the fabricated
  citation — i.e. the escape hatch is closed, and visibly so.

Harness validation

Both parser sets were pointed at a known-bad state before any green was
trusted: a path that does not exist, and a real file of the wrong shape. Both
raise HARNESS BROKEN / ENOENT rather than returning an empty set — checked in
as test_parsers_fail_loudly_on_a_missing_source and
test_parsers_fail_loudly_on_a_source_of_the_wrong_shape, with
test_the_parsed_inventory_is_non_empty_and_plausible as the positive control.
M21/M22/M23 then sabotage the live parsers and confirm the guards fire.

Measured while writing the SKILL.md parser: splitting table rows on raw |
silently drops the emulate and key rows (both carry \| inside a backtick
span), which would have made a documentation test fail for entirely the wrong
reason. The parser splits on unescaped pipes only; M23 pins that.

Not done / flagged

• No existing assertion was weakened or deleted; no production file was touched.
  Files owned by sibling agents (browser, server.py, SKILL.md,
  test_skill_size.py, reference/**) are untouched.
• The .mjs suites are outside the flake gate (above). Worth a follow-up.
• pytest at HEAD is fully green; the one red test is in the node suite.

🤖 Generated with Claude Code

[ZacxDev]

Parser hardened — a cmd_op MENTION is no longer a dispatch (0f5577f)

Thanks for the cross-PR catch. Fixed as a class, not by special-casing stderr. Also rebased onto current main (556aa1b, includes #266/#272/#274/#275/#279) — rebase, not merge; clean, no conflicts.

The rule now

Two independent conditions, both required:

1. the occurrence survives mask_shell_noncode() — a small shell lexer that blanks comments, single-/double-quoted strings (including multi-line), backquoted spans and heredoc bodies, preserving byte offsets. $( … ) is deliberately not masked even inside double quotes: that is genuine command position.
2. it sits in command position — start of statement, or after ; && || | ( $( then do else.

New fixture tests/fixtures/cmd_op_parse_rig.sh pins 8 dispatch shapes that must be harvested and 11 mention shapes that must not be.

Red-first

Against the rig, the previous parser harvested 7 phantoms while keeping all 8 real dispatches:

phantombacktick, phantomdocstring, phantomdq, phantomheredoc,
phantommultilinedq, phantomquotedheredoc, phantomsq

Mutation P1 (revert to the old parser) reproduces exactly that list, so the demonstration is checked in and repeatable rather than a one-off claim.

Against the real #278 shape — that exact docstring injected into a throwaway copy of browser (the real file is yours and is untouched): stderr not harvested, op set 19 → 19.

Over-tightening controls

A permanently-green no-op is the obvious wrong fix, so:

• the rig set must match exactly — misses are named, not just leaks;
• the real CLI must still yield all 19 ops, asserted == server.py's inventory;
• screenshot is pinned by name. It is dispatched from resp="$(cmd_op screenshot "\$full")", a command substitution nested in double quotes. An earlier version of my masker blanked the $( and lost exactly that one op, 19 → 18, while every other test in the file stayed green. This control caught it, not review.
• new test pins that the CLI has no live backtick substitution, since backquoted spans are masked conservatively (113 backticks, all inside comments — measured).

Mutation sweep — 10 mutations, all red, each naming its specific phantom/missing op

Two survived the first pass, and are why the rig grew two more cases:

#	mutation	first pass	after rig fix
P5	drop command-position anchoring	survived — masking alone already caught every phantom	✔ red on phantombareword (unquoted echo usage: cmd_op …)
P6	drop single-quote masking	survived — green for the WRONG reason: the """ in the docstring case toggled double-quote state, so a different defence was catching it	✔ red on phantomsqblock (single-quoted python3 -c body with no double quotes)

Other mutations: P1 revert-parser ✔, P2 blank $( again ✔ (red naming realsubshell and screenshot), P3 tighten to a no-op ✔ (HARNESS BROKEN, not vacuous green), P4 drop heredoc masking ✔, P7 M14 redux — new wire op sniff in server.py+protocol.js only ✔ still red naming sniff, P8 delete a real rig dispatch ✔, P0 identity control reproduces the baseline exactly.

Counts

Counted from reporter lines. pytest 9.0.2 confirmed importable first — noting for the lineage that a bare python3 -m pytest on this host exits 0 printing No module named pytest.

suite	before	after
pytest	381 passed, 0 failed	385 passed, 0 failed
node	460 tests / 459 pass / 1 fail	460 / 459 / 1 — unchanged

That one node failure is still the deliberate AGENT SURFACE PARTITION finding on context, emulate, ping. Not pre-empted — it stays failing until the operator rules. browser and server.py untouched.