[mbreiser]

Summary

Lessons from the first two releases, turned into tooling and docs:

1. pixi run test discovers tests by name (scripts/run-tests.py): every tests/test-*.js|py and tests/validate-*.js. It replaces the hand-kept && chain in pixi.toml. Every PR that added a test appended to that same line, so every release candidate hit the same merge conflict (candidate 2 hit it three times). Details:
   • Each file runs in its own process, and every Node test gets the vendored yaml hook, passed as a file URL so it also works on Windows.
   • Every file runs even after a failure; a summary follows, and the exit status is non-zero if anything failed.
   • pixi run test -- wire runs a subset, and -- --list shows what would run.
   • It picks up 8 files that were running nowhere: test-crc.js, validate-arena-geometry.js, validate-header-v2.js, validate-matlab-reference.js, and the four validate-* suites only their own workflows ran. All pass. Total: 56 files, about 15 s.
2. A bench validation kit.
   • protocols/validation/bench_closed_loop_2x10.yaml, the closed-loop check used for feat(closed loop): a trial opens at its trialParams frame_index — Studio v0.91 #225 and both 2026-09-2x candidates. It needs the CSHL course pattern bundle on the SD card.
   • pixi run bench-check -- LOG --hold epoch_a_frame25=25 --hold epoch_b_frame75=75 --channel next --build <sha>. It checks that the run completed, every arena command was ok, there were no error glyphs, and the closed loop held its frames. It prints Studio version, tier, build, controller firmware and panel inventory.
   • It reads behavior_v1 and v2 logs, plain or .gz. I verified it on the real 2026-09-28/29 bench logs (all pass), and it has its own tests with synthetic logs.
3. Release docs and the g6-release skill:
   • An adversarial review of the candidate diff before any bench time. On 2026-09-29, rc1 and rc2 were validated first and then had to be redone after the review held Assert sticky controller settings before every run: controller: block, rig defaults, run header (LAB-230) #219 back.
   • The bench kit steps, including turning the course-repo upload off for validation runs, since the lab browsers have it on.
   • Keep PRs mergeable: GitHub runs no CI on a conflicting PR, which is what happened to Studio v0.93: panel inventory — which panels are present and their firmware, in every run log #233.
   • New tests need only the naming convention.

Linear: LAB-222 (two-tier web releases)

Tier

Tooling and docs only, merged after CI like #232. No served page changes: a new YAML file under protocols/validation/ is served but not listed in protocols/index.json.

Release notes

None (no tool changes).

Test plan

• pixi run test: 56/56 files pass (the old 46 + 8 previously unrun + 2 new)
• tests/test-run-tests.py 12/12, tests/test-bench-check.py 20/20
• pixi run bench-check on the rc3, Production v0.92 and v0.91-baseline logs: all pass, including the behavior_v1 log
• The validation protocol passes the protocol validator
• CI green, including the 3-OS tiers job

🤖 Generated with Claude Code