Skip to content

LAB-303: ROS 2 bridge graph + MCAP record, ScanImage/BIAS connectors, Studio v0.97 (candidate 2026-10-20) - #254

Open
mbreiser wants to merge 102 commits into
mainfrom
claude/ros-studio-plan-b-c44942
Open

mbreiser wants to merge 102 commits into
mainfrom
claude/ros-studio-plan-b-c44942

Conversation

@mbreiser

@mbreiser mbreiser commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

Summary

LAB-303 — the ROS 2 graph becomes the rig bridge, with MCAP as the record. Decided 2026-10-16 (Frank approved) on the measured results of this branch: Linux container, the Mac bench, the Windows rig PC ww18 (2026-10-15, real FicTrac + BIAS, 10-min soak, idx mismatch 0.00 %) and the Bergamo 2P rig (2026-10-16: a real color-sweep experiment, 26/26 steps, ScanImage driven from the protocol, 0 dropped frames). Constituent 3 of candidate 2026-10-20 (with #257 and #256); it ships through Next → Production like every other PR. Bridge 3.4 stays installed side by side for course rigs; the browser protocol is unchanged.

What is in it (numbers and details: ros2-poc/README.md, ros2-poc/windows-results-ww18-2026-10-15.md, ros2-poc/scanimage-day-2026-10-16.md):

  • ros2-poc/ — the bridge as a ROS 2 graph (RoboStack Jazzy + rmw_zenoh, its own pixi workspace): pixi run rig supervises the router, the FicTrac adapter, the bridge node and optional device nodes. The bridge node reuses fictrac-bridge/bridge.py's Hub / LogWriter / Pipeline / dispatcher unmodified, so the Studio and every reader see the same protocol and the same behavior_v2 file.
  • MCAP as the record — nodes/recorder.py (rosbag2_py + MCAP plugin): one bag per run with every device topic as received, /arena/frame with the node's own sample time, /arena/browser, /arena/event, /arena/session; crashsafe profile by default (a SIGKILL loses ≈ 1 frame; rosbag2 defaults lose a whole short run). nodes/mcap_convert.py derives the .jsonl (byte-identical to the live writer for every sequenced line); nodes/mcap_index.py reindexes + compresses a closed bag (atomic replace) and recovers a torn one; a restarted node recovers the newest unclosed bag on log_export. Optional --max-bag-mb splitting (off by default — bags are archived on a share, not pushed to GitHub).
  • Device connectors driven from the protocol: BiasPlugin → nodes/bias_node.py (BIAS HTTP), ScanImagePlugin → nodes/scanimage_node.py (ScanImage's built-in JSON server, allowlisted get/set, two fixed evals; chosen over si_zmp on LAB-305's measurements). The runner sends connector_command, the client waits with the step's timeout_s + a margin, outcomes are ok | error | unknown; a ≤ 3.4 bridge yields a visible skip. First 2P protocol: protocols/g6_2p_color_led_sweep.yaml (ran the 2026-10-16 dataset).
  • Typed interfaces (interfaces/arena_interfaces: FicTracFrame, ArenaFrame, ConnectorReply) built by pixi install -e typed (pixi-build-ros) or taken from the CI-built channel (.github/workflows/arena-interfaces.yml, three platforms); the default environment stays String-only so any rig PC installs.
  • Seeing the graph: pixi run graph, the viz environment (rqt, foxglove_bridge), --foxglove runs the typed republisher nodes/viz_node.py (ported from ros2-poc: ROS 2 bridge proof of concept (LAB-303) #252) + foxglove_bridge; layouts in foxglove/; nodes/foxglove_json.py makes a String-topic bag plottable offline.
  • Studio v0.97 (on top of Studio: stop the run when the FicTrac bridge is lost (BRIDGE_LOST) + ?p= links never wait on GitHub (LAB-303 S1, S2) #257's bridge-loss and ?p= hardening): the ScanImage plugin in the registry, connector commands in the Run log, connector plugin instances counted as bridge-dependent steps, the bridge log closes at run end (also the bag), list-style variables: no longer produce a false duplicate warning, ?v= cache tokens bumped on the changed modules.
  • Tooling kept inert: scripts/tiers/stamp.py --ros REF (an optional /ros/ side path; off by default, not used by the candidate).

Review record: .codex-review/report-20261017-183000.md (Codex cross-review of the rollout plan) + the 2026-10-17 code review findings, all addressed here or in #257.

Tier

next candidate — 2026-10-20, constituent 2 (after #256). #257 (BRIDGE_LOST + ?p= + Push-log guard) is contained in this branch and ships through it; its release notes are folded in below. Bench gates before promotion: T1 ball rig (ww18: closed loop + BIAS, bridge kill → BRIDGE_LOST, Push log refuses the wrong file, bag reindexed) + T1b bridge 3.4 compatibility, T2 Bergamo (Chrome 156 local-network prompt, Inventory 20/20, one connector grab, 2P dry run, bag closes without a token).

Release notes

Studio

  • Losing the FicTrac bridge mid-run now stops the run and says so. If the bridge closes during a closed-loop run, a run with FicTrac steps, or a recorded experiment, the run stops (the display gets STOP, the closed loop turns off). A red alert stays on screen until you dismiss it, and the run log records the outcome as BRIDGE_LOST. A test run that needs nothing from the bridge keeps going, and its log notes that bridge logging stopped.
  • A shared protocol link (?p=…) opens right away, even when GitHub is slow. It used to wait for the course repo's pattern list from GitHub and could stay empty until you refreshed. Pattern thumbnails still appear as soon as GitHub answers.
  • Run-log pushes can no longer commit the wrong file after a bridge restart. ⇪ Push log checks that the bridge serves this run's own log file and refuses otherwise. It tells you where the file is on the bridge PC.
  • A protocol can drive ScanImage (two-photon rigs). Declare a scanimage plugin and use Set file stem, Grab, Wait until idle, Abort and Set property (a few allowlisted settings) in conditions; the commands go through the ROS bridge node to the microscope's JSON server, and every command and reply is in the run log with the TIFF stem and counter. Web runner only.
  • Camera (BIAS) commands in a protocol act for real on the ROS bridge instead of being skipped: connect, start/stop recording, timestamps; the run log shows each outcome, round trip and the recording path. A bridge that cannot drive devices says so in the Run log rather than skipping silently.
  • The bridge's run file closes when the run ends, also for runs that are not committed (no course token, Test runs, aborts). On the ROS bridge node this closes the MCAP bag; "⇪ Push log" still exports the most recent file.
  • Camera and microscope steps wait a little longer than their own timeout_s before being called unknown, so a reply that lands right at the deadline counts.
  • Protocols that list their variables (- &name value) no longer trigger a false "Variable … is defined more than once" warning.

ROS bridge (new; rig PCs)

  • cd ros2-poc && pixi install && pixi run rig replaces pixi run bridge on rigs that move to the graph: same WebSocket, same run log for the Studio and the dashboard, plus an MCAP bag per run as the record (Foxglove opens it), optional BIAS and ScanImage device nodes, and --foxglove for live dashboards. Bridge 3.4 keeps working for course rigs.

Test plan

  • cd ros2-poc && pixi install --locked; pixi run test harness (default env) — all checks PASS (crash-recovery phases included); pixi run test-viz PASS
  • root pixi run test — 70/70 files (runner 547, client 199, session 106, adapter 34, run-log 39, studio-shell 82, preview-sources 23, v3 round-trip incl. the new list-style variables suite); Prettier clean on touched JS; tests/test-tiers-candidate.py 109/109
  • Mac bench E2E (2026-10-14): Studio + G6 controller + real BIAS through the graph, bag converted byte-identical, closed-loop-report / runlog-check OK
  • Windows rig PC ww18 (2026-10-15): real FicTrac + BIAS, 5.7 bridge kill → every frame recovered, 10-min soak 0.00 %
  • Bergamo (2026-10-16): the 2P color sweep through the ScanImage connector, 26/26 steps, 13/13 connector commands, 0 dropped frames
  • T1 / T1b / T2 on the 2026-10-20 candidate from Next (see Tier)

LAB-303

🤖 Generated with Claude Code

mbreiser and others added 30 commits October 2, 2026 14:57
The Program set confirm said ~8 min for a 20-panel arena; the batch on
rig5-mr (2026-09-30) took ~2 min: ~4.4 s program + reboot and ~1 s
verify + STOP per panel. Use 5 s + 1 s for the estimate and fix the
"~20 s" in the dialog and the Flash / Program set tooltips.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ial arenas

getDegreesPerPixel() divided 360 deg by the INSTALLED pixel columns instead of the
full circumference. On G4 3x12of18 (the IAS talk arena) that gave 1.875 deg/px
instead of 1.25, so the px<->deg helper labels were wrong and a spatial period typed
in pixels generated the wrong grating (24 px came out as 45 deg, 36 frames). A step
size typed in degrees was off the same way. G6 3x12of18 (1.5 -> 1.0 deg/px) and
G6 2x8of10 (2.25 -> 1.8) had the same bug; full arenas were already right.

The generator already used the full circle (arenaCoordinates numCircle = num_cols).
The pitch now lives in one shared place, PatternGenerator.getDegreesPerPixel(), and
getArenaDimensions() reports circleCols / azimuthPixels, which the three spherical
generators also read, so the editor and generator cannot drift apart again.
Also fixes the Animate tab's frame-shift hint on partial arenas ("360 deg = 192 px"
-> "240 deg = 192 px"), and cache-busts generator.js because the editor's init now
calls the new helper.

tests/test-pattern-degrees-per-pixel.js runs the editor's own conversion functions
(extracted from pattern_editor.html) for every standard config, plus end-to-end
gratings typed in px and deg; it failed 15 checks before the fix.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…olumn

A partial arena's PAT stores only the installed columns, in columns_installed order
(block k = physical column columns_installed[k], the icon generator's convention).
ThreeViewer looked pixels up by the PHYSICAL column instead. Harmless when the
installed columns start at 0 (G4/G6 3x12of18), but on G6 2x8of10 (columns 1-8)
the whole pattern showed one panel to the side, pattern panel 1 never appeared, and
physical column 8 read block 0 of the next pixel row (the lookup wrapped at the full
circle, past the 160-px pattern width).

_getPatternColumnMap() maps physical column -> block from the CONFIGURED columns
(never the preview-visible set, so the CSHL 8/18 gap still doesn't shift the address
map); each LED ref carries patternCol; the lookup wraps inside the pattern width.
Shared by the Pattern Designer and the replay viewer, so the cache token moves to
1006-partial-cols in both (and the tests that pin it).

tests/test-three-viewer-partial-columns.js loads the viewer with Three.js stubbed and
checks every LED of every standard config reads its own block and row; against main's
viewer it fails only on G6_2x8of10 (column 1 showed block 2).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
renderCylindricalIconToCanvas drew math angles (y up) straight onto the canvas (y down),
so every icon was the 3D "Top Down" view mirrored: the gap (behind the fly) at the TOP and
the columns running counter-clockwise. TESTING.md always expected the gap at the bottom.
Angles are now negated at the drawing step (toCanvas), so icons match the 3D viewer and
MATLAB design_arena: south at the bottom, column 0 at the south boundary, columns clockwise
from above. The misleading "row 0 = top of arena" comment is corrected (row 0 is the
bottom; it maps to the inner ring, as in a perspective view from above).

Shared by the Studio, Console and Pattern Designer thumbnails. No export changes.
tests/test-icon-orientation.js records the canvas arcs and checks column 1 and 2 of every
standard config against the 3D viewer's placement; the old code failed 21 of 27 checks
(G6 2x10 column 1 drawn at -18 deg instead of -162 deg).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…zimuth axis

Panel numbers: the 2D overlay labelled pattern blocks as if they were physical columns and
looped over every num_cols (drawing labels past the right edge on partial arenas). It now
labels each block with its physical panel through physicalPanelNumber() (block k =
columns_installed[k], G6 row-major / older generations column-major), the same numbers as
the 3D overlay: on G6 2x8of10 the first pattern panel reads 2/12, not 1/11.

Azimuth axis: a degree scale under the 2D grid (its own canvas, so edit-mode hit-testing
and captures are unchanged), in pattern coordinates: 0 deg = straight ahead at the middle,
negative to the fly's left; same values as the Mollweide view and the generator. Ticks
pick 15/30/45/60/90 deg to keep labels apart (G4 3x12of18: -120..+120 every 30 deg).

CLAUDE.md gains the three partial-arena rules (full-circle pitch, block k =
columns_installed[k], 0 deg = middle of the pattern) and points at LAB-296 for hardware.
tests/test-pattern-2d-view.js runs the extracted helpers against every standard config,
the 3D overlay's numbering and the Mollweide longitudes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…F color code

Color variants of one grating used to share a name (the filename carried only the layout
tag), and re-saves stacked tags (_4c_rir). Names now:
- record the ON/OFF choice used to GENERATE the frames, stamped at generate time:
  PanelColor.filenameColorCode -> '_4c-B' (ON blue), '_4c-B-G' (ON blue, OFF green),
  '_4c-VB', '_4c-F80A' (custom levels, one hex digit per channel); nothing extra for the
  default (ON all / OFF dark), so default names are unchanged apart from the words below;
- normalize the layout tag on save (PanelColor.stripColorTag): replaced, never stacked;
- use short words: grat/star/edge/sine/offon/anim, rot/exp/trans, Ndot, _img, comb,
  hcw/hccw/vup/vdown.

The controller keeps pattern names in a 64-byte buffer: <= 63 characters, plain ASCII,
and <= 58 so a same-name re-upload (stored as X001_<name>) keeps its .pat. Nothing in the
Designer checked this. js/pattern-set.js gains the shared rule (checkSdFilename,
sanitizeSdFilename, fitSdFilename; both tools load it), and the Designer fits every saved
and renamed name (fitPatternFilename: cleans characters, shortens only the descriptive
middle, keeps the arena prefix and the color tag; the rename prompt says what it changed).
The longest name Generate can produce is now 54 characters with the worst-case tag.

Helpers are typeof-guarded, so a stale cached panel-color.js / pattern-set.js only means
the old naming until a refresh.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ads, 0x85 label

- sdUploadOne runs PatternSet.checkSdFilename BEFORE sending the bytes: a name the
  controller can't store (non-ASCII, > 63 chars, no .pat) fails at once instead of after
  the whole upload (which left the data on the card as pattern.temp), and is not retried.
  59-63 characters log a warning (a re-upload would lose .pat).
- An upload never overwrites: the firmware stores a same-name re-upload as X001_<name>.
  sdLogicalName now strips that prefix, so the copy shows as a DUPLICATE of the original
  (onSdListing warns; protocols refuse to guess) instead of a separate pattern while the
  picker kept playing the old file. The upload also warns up front when the name is
  already on the card.
- The upload opcode is SET_PATTERN_FILE 0x85; the Studio's error text ("write rejected"),
  tooltips and comments said 0x8D, which the firmware doesn't have (also fixed in the
  Console comment and CLAUDE.md's quiesce list).

CLAUDE.md records the filename rule. tests/test-sd-filenames.js covers the shared rule,
the Designer's names (every arena prefix + worst-case tag <= 58), withColorTag /
fitPatternFilename, sdLogicalName and the upload order.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…l-G6 warning

From the adversarial review of candidate 2026-10-06 rc1 (Claude + Codex):
- The shared SD name rule (PatternSet.checkSdFilename) refused anything outside
  [A-Za-z0-9._-], so the Studio blocked uploads that worked before: spaces and
  parentheses are legal FAT long-name characters (e.g. a browser's "… (1).pat").
  It now errors only on what the card can't store (non-printable/non-ASCII, \ / : * ? " < > |,
  > 63 chars, no .pat, leading "."); the Designer still writes tidy names
  (SD_TIDY_CHARS) and says so when it changes one.
- Cache tokens (?v=1006-partial-cols) on every consumer of the changed shared modules:
  icon-generator.js (Designer, Studio, Console, v3 designer, icon tool), pattern-set.js
  (Designer, Studio, Console, v3 designer), panel-color.js (Designer) — so the fixes reach
  browsers holding an old copy. Pinned in tests.
- Partial-G6 warning (never blocks): current firmware ignores the panel mask and rejects
  any grid other than its own at load. The Designer says so once per session when a
  partial-G6 pattern is saved; the Studio upload warns when the session rig is a partial
  arena, when a file carries a partial panel mask, or when its grid differs from the rig's
  (PatternSet.g6HeaderGrid / g6PlaybackWarning; Studio.rigArena()).
- 2D panel numbers: a block past the installed list labels its own number, never NaN.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…he thumbnail download (LAB-298)

Bench 2026-10-07 (rig cshl_g6_2x10_ball, 24 patterns on the card): the first
TRIAL_PARAMS after Connect was acknowledged 57 s late. The SD listing auto-picks
pattern 1 and the picker thumbnail downloaded the whole file (0x84). With
Chrome's default 255-byte Web Serial read buffer the body arrived 901 bytes
short, and the one-at-a-time link waited out the read's 60 s timeout with the
GET_PATTERN_INFO sweep and the user's command queued behind it. Firmware ruled
out from Python (listing + 24x 0x88 in 48 ms, 81 KB download in 6 ms).

- arena-link.js: open the port with a 1 MiB bufferSize; a background lane
  (`{ background: true }`) that waits while any foreground send is queued; a
  bulk body fails after idleTimeoutMs (5 s) without a byte instead of waiting
  out timeoutMs.
- Studio: the GET_PATTERN_INFO sweep (cardPatternInfo, also caches file size)
  and the picker thumbnail fetch use the background lane; a card-only
  thumbnail downloads only when PatPreview.liveFetchVerdict allows it (size
  known, <= 1 MiB, no run). Frame counts for the closed-loop resolver (#201)
  are still filled for every pattern. Tooltip explains a thumbnail refused
  while the display runs (status 10).
- Tests: link buffer/stall/lanes, liveFetchVerdict, Studio call sites.

Verified on hardware: first TRIAL_PARAMS ack 2 ms (was 57,013 ms); 81 KB
thumbnail download 7 ms; 24 frame counts done 197 ms after Connect.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ipherals (LAB-214, Studio v0.96)

- js/arena-wire-g6.js: GET_I2C_SCAN 0xB0 / I2C_TRANSFER 0xB1 encoders + decoders
  (status text, "No Qwiic jack" detection) with golden tests.
- js/qwiic-drivers.js (new): host register maps ported from the firmware repo's
  tests/qwiic_sensors.py + qwiic_peripherals.py — SHT4x (two-leg read, CRC),
  LSM6DSOX (tilt, latched bump: LIR + INT_CLR_ON_READ + INT1 routing), TSL2591,
  VEML7700, AS7343 (releases the bus while integrating), PCA9685 (ALLCALL off,
  servo pulse), PCA9548; an async bus mutex; all-call shadow decided from MODE1;
  the KINDS registry is the one place to add or drop a part.
- js/studio-qwiic.js (new): rig `peripherals:` parser (tolerant, warnings), the
  scan-vs-YAML check in both directions (declared-but-missing = error,
  attached-but-undeclared = warning), setup, chip, "Copy as rig YAML".
- arena_studio.html: one Console "Qwiic" window (Scan → device dropdown → one
  device's readout in display units; bump check; servo go-to-position), a
  connect-time setup scan and a top-bar chip. Every I2C command is gated on the
  0xC2 qwiic_i2c feature bit: controllers without the jack are never sent any.
  Test only — nothing is written to run logs (that is PR 2).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…nection columns, no sticky "live" status (LAB-214)

Bench feedback (all six boards working on the 12-18):
- The scan table is the picker: one radio row per identified device (click the
  row or its button); the dropdown is gone.
- Each row names its category (Servo / PWM, Temp / humidity, Accel / tilt / bump,
  Light (broadband / lux / spectral), I2C mux) — KINDS.category, one place.
- "where" is now "connection": direct (on the controller's own chain) or
  "mux 0x70 · ch N" (behind a PCA9548 channel).
- Stopping Live restores the scan summary; a tick in flight at Stop no longer
  re-writes "live · N Hz".
- Table text left-aligned (it inherited the analog table's right alignment).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…-214)

- js/arena-configs.js regenerated (scripts/generate-arena-configs.js) from the
  maDisplayTools branch feat/arenas-1x12of18-3x10of14: G6 arena IDs 7
  (G6_1x12of18) and 8 (G6_3x10of14). angle_offset_deg centers the gap at the back
  so the installed panels sit centered at the front in the 3D view and icons
  (Mercator/Mollweide center the installed arc by construction).
- configs/rigs/g6_1x12of18_bench.yaml: the arena_12-18 bench, one row, with the
  six-board Qwiic chain declared under peripherals:; g6_3x10of14.yaml with
  peripherals: [] until its chain is known. Both added to configs/rigs/index.json.
- tests: geometry, IDs, centering, index entries, clean peripherals parse.

The maDisplayTools change must merge first: the sync-arena-configs workflow
regenerates arena-configs.js from maDisplayTools main.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s, color-bank figure; Qwiic read-on-select (LAB-214)

- Qwiic: clicking a device reads it immediately; Live keeps running across device
  changes (each tick reads the selected device); the servo follows its slider while
  dragging.
- Console sliders act while dragging — Test display level/duty re-send the last
  figure, LED power sets the LED — through liveSender(): single-flight, latest
  value wins, nothing sent while a run owns the link, one log line on release.
- Test display streams at the CONTROLLER's frame geometry (0xCB rows×cols; fw
  panel_count_per_frame) instead of refusing every non-20-panel rig: the rig figure
  is placed in the firmware frame (installed columns from the left, rig rows
  repeated up), so the 1×12 bench drives a 4×12 build. Wire.streamFrameBytes +
  encodeStreamFrame(payload, {panels}) with golden tests; .bin/.pat file streaming
  stays 20-panel.
- Test display → color A–D: light one LED bank (T0–T3) at a time via the shared
  PanelColor.bankAt (100 of 400 LEDs per panel).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…commit bumps; notes live in the PR body

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…n), live gates, deferred Qwiic setup (LAB-214)

From the adversarial review of candidate 2026-10-06 rc4 (PR #247 comment):
- BLOCKING: test figures are drawn on the rig's FULL arena grid again
  (activeArenaGeom = num_rows × num_cols); grid column c → firmware column c and
  only the installed columns must fit the firmware frame. rc4 collapsed partial
  arenas to installed width and left-aligned them, which shifted every figure one
  panel on G6_2x8of10 (columns 1–8) with 0xCB firmware and refused it outright on
  pre-0xCB firmware; it also mis-numbered the ISP progress map and changed the
  paste dimensions. Mapping now lives in js/studio-stream-geometry.js with Node
  tests (2×10 / 2×8of10 identity on both firmware kinds, 1×12of18 → 4×12).
- Live slider re-streams and Qwiic reads pause during an ISP batch and a
  post-mortem (a stream re-activates the display; 0xC8/0xC9 need it stopped).
- Connect-time Qwiic setup starts after the controller reads and is not awaited,
  so its sensor configure waits never delay the SD listing.
- Orientation glyphs sit inside the installed span (1×12of18's "R").
- Best-effort servo release (PCA9685 all off) on an intentional disconnect.
- Docs: drop the stale "v0.96" (the release commit owns versions).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…apping, unmirrored icons, short SD-safe filenames, upload name checks (LAB-295)
…ve Console sliders, firmware-sized test display + color banks (LAB-214)
…1.5, Pattern Designer v0.13, Arena Studio v0.95

Candidate for Production. PRs: #245, #246, #248, #250.

Release-Manifest: releases/2026-10-06.json
…ters, bridge node reusing bridge.py

Separate pixi workspace (RoboStack Jazzy + conda-forge; osx-arm64/win-64/linux-64) under
ros2-poc/; the root pixi.toml and fictrac-bridge/ are untouched so bridge 3.4 stays side by side.

- Zenoh router + rmw_zenoh_cpp with loopback-only configs (stock files, endpoint lines only)
- fictrac_udp_adapter: FicTrac UDP → /fictrac/raw (std_msgs/String JSON), FicTrac untouched
- kinefly_udp_adapter + kinefly_sim: KF/1 (mbreiser/Kinefly-MacOS) → /kinefly/raw
- bias_node + bias_sim + bias_cmd: BIAS HTTP command port ⇄ /bias/cmd,/bias/reply,/bias/status;
  timeout = unknown outcome + reconcile; video file reported as an artifact
- bridge_node: bridge.py's Hub/LogWriter/Pipeline/make_dispatcher on ROS input, same WebSocket
  schema on ws://localhost:8765, same behavior_v2 log; adds connector_status latency lines,
  source_lost/resumed watchdog, ["kf",…] rows, BIAS connector events, /arena/frame mirror
- rig.py: one `pixi run rig` command supervising router + nodes
- tests: studio_emulator (the Studio runner's exact bridge conversation), check_runlog_readers
  (JS readers accept the log), test_ros2_poc (29 checks incl. adapter SIGKILL mid-run,
  closed-loop report rejection/idx checks, convert round trip): 29/29 on Linux

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KHyggowbCA5ucBg71bU26V
…outer restart, launcher pre-flight)

- bridge_node: RosLogWriter hooks (on_open/on_close) replace the message re-parsing wrapper;
  source watchdog scoped per log file; ordered ROS shutdown (join the spin thread before
  destroy_node — removes "terminate called without an active exception" on SIGTERM)
- bias_node: worker exception guard (a malformed BIAS body no longer kills the node),
  get-status polls coalesced to one pending
- rig.py: port pre-flight with a one-line culprit per port (SO_REUSEADDR probe), router death
  stops the rig, Zenoh config paths set explicitly (Windows shells leave $PIXI_PROJECT_ROOT literal)
- nodes/_common.py: shared envelope, QoS, spin/shutdown boilerplate, wait_port
- test harness: adapter kill anchored on the runner's fictracApply line in the log; export guard
- README: soak (12 runs, flat RSS, p99 ≈ 3.5 ms), zero loss at 200 Hz + 500 Hz, router
  kill/restart with no data gap, student-mistake messages

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KHyggowbCA5ucBg71bU26V
… TerminateProcess)

Python cannot send SIGINT to another process on Windows (send_signal raises ValueError), so
Ctrl-C on the rig PC would have looked like a launcher crash. popen_group/signal_group in
nodes/_common.py give every child its own process group on both platforms and map
SIGINT → CTRL_BREAK_EVENT, SIGTERM/SIGKILL → TerminateProcess on Windows; rig.py and the
test harness use them (and catch ValueError). Linux path unchanged (setsid + killpg).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KHyggowbCA5ucBg71bU26V
…s a watch item)

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KHyggowbCA5ucBg71bU26V
Mac run of the items the Linux container could not cover: pixi install 25 s
fresh, talker/listener, 29/29 acceptance, the unmodified Pages Studio against
`pixi run rig -- --sim` with a G6 controller (two Mode-3 constant-bias runs
auto-committed to the course repo, byte-identical to the bridge node's file,
idx mismatch 0.00 %, runlog-check OK), the real BIAS Mac port and the real
Kinefly fork (9600/9600 datagrams, 0 rejected, kf rows in the committed log).

bias_node: the real Mac port resolves a relative `filename` against its own
cwd and start-capture then succeeds without capturing; the reply now carries
a warning for relative names.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
WINDOWS-TEST.md scripts the one open acceptance item (install timing, the
$PIXI_PROJECT_ROOT expansion check, talker/listener, the 29-check harness,
rig + unmodified Studio + controller, real BIAS over HTTP with an absolute
video path and the unknown-outcome path, Ctrl-C cleanup, student mistakes)
and lists what to send back.

Windows' signal module has no SIGKILL, so rig.py's stop escalation, the
harness's mid-run adapter kill and _common.signal_group would have raised
AttributeError there; they now share _common.SIGKILL (getattr fallback 9).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ectors)

The web runner executed only the `log` and FicTrac plugins; every other plugin
command was a visible skip, so a protocol's `camera` plugin (BiasPlugin) never
recorded in a web run. Now:

- js/arena-runner-g6.js: a `connectorCommand` op for plugins named in
  `connectorPluginNames` (instance name → connector id, 'bias' for BiasPlugin).
  The step waits for the bridge's reply; ok → command event with the reply
  (outcome, value, artifact, rtt); error/unknown → step error, run continues
  (proceed-and-skip). A bridge that advertises no connectors (every ≤ 3.4
  bridge) yields a skip that names the bridge version. listSkippedPlugins
  takes an optional driveable set so the Edit-view note counts real skips.
- js/fictrac-bridge-client.js: connectorCommand() → {type:'connector_command'}
  and the id-matched connector_reply; never rejects (not connected → error,
  no reply in time → 'unknown', disconnect in flight → 'unknown'). hello_ack's
  `connectors` list → supportsConnector(). js/arena-session.js passes the map.
- arena_studio.html: connectorPluginNames(exp) from BiasPlugin instances,
  passed to runSequence; Run log lines show outcome · rtt · artifact; footer
  v0.95; release notes.
- ros2-poc bridge_node.py: hello_ack gains `connectors: ['bias']`;
  connector_command → /bias/cmd, the matching /bias/reply → connector_reply to
  the asking websocket; both halves logged (`via: ws` on Studio-path replies).
  bias_node.py: --video-dir resolves a relative filename (MATLAB's
  experiment-folder behavior). Emulator drives connect/startRecording/
  getTimestamp/stopRecording/bogus through the bridge; harness 32/32.

Tests: runner 512/512, bridge client 173/173, root suite 60/60 files.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…95 connector path)

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…erter, ROS Studio tier (LAB-303)

Frank's Plan B built as a bounded spike on the existing ROS 2 PoC graph, so the LAB-303
architecture decision can be made on measured data (ros2-poc/README.md → "Plan B spike").
Bridge 3.4 stays installed side by side; the browser protocol is unchanged.

* nodes/recorder.py + bridge_node.py: rosbag2_py records every topic per run into an MCAP
  bag — device topics as received, /arena/frame with the closed-loop node's own sample time,
  /arena/browser (every browser message, verbatim), /arena/event, /arena/session — each
  bridge-side record with the writer's emission sequence number. Profiles: crashsafe
  (default; no chunking, no cache) and rosbag2's default (one unflushed 768 KiB chunk).
* nodes/mcap_convert.py (`pixi run convert`): bag → behavior_v2 / behavior_v1 / full NDJSON,
  pure Python (mcap + a schema-driven CDR decoder, no ROS process), `--recover` for a torn
  bag, `--order seq|time`. log_export returns the converted log; the live NDJSON stays as a
  shadow during the spike and `converted == live` is checked at every export.
* interfaces/arena_interfaces (FicTracFrame, ArenaFrame, ConnectorReply) built by
  `pixi install` through pixi-build-ros; `pixi run build-interfaces` = the colcon alternative
  (install rpath fix for macOS). The adapter publishes /fictrac/frame typed; the bridge
  consumes it and publishes/records /arena/frame typed when the package imports.
* tests/test_ros2_poc.py: judged against the CONVERTED log; + bag per topic, fidelity vs the
  shadow (sequenced lines byte-identical; derived lines ±2 ms), levels, ordering, typed
  schemas, and a SIGKILL crash phase for both profiles. 46 / 46 on the Mac.
* ROS Studio Pages path: scripts/tiers/stamp.py --ros REF → /ros/ (channel "ros", magenta
  banner), deploy-pages.yml input `ros` (off by default), `pixi run ros-preview`,
  js/build-channel.js channel "ros" (+ badge, window-name suffix, tests). Production and
  Next untouched.
* protocols/bias_camera_closed_loop_poc.yaml (+ index entry): the bench protocol.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
mbreiser and others added 15 commits October 16, 2026 12:49
…n week, item 1)

A bag whose writer was killed (no metadata.yaml; a torn tail) no longer needs
`--recover` by hand. mcap_index.recover_bag rewrites the .mcap whole + indexed
(torn tail dropped, original kept as .torn.bak), appends one /arena/session
{"event":"recovered"} record so the bag itself says so, and asks rosbag2's own
Reindexer for metadata.yaml (verified: it reads the crashsafe file in file
order). The bridge node runs it when a log_export meets such a bag; a freshly
started node adopts the newest unclosed bag in --log-dir for its first export,
so the Studio's export after a bridge restart gets the salvaged run. The
log_export_result carries `recovered: true` + the recovery report (and the
conversion report); the converter emits the marker as a session line; a bag
with nothing salvageable (rosbag2 defaults leave a 0-byte file) falls back to
the shadow NDJSON. `pixi run export -- --json`, `pixi run reindex -- BAG --recover`.

Harness crash phase: a restarted node's log_export recovers the bag on its own
(frames == the --recover CLI's, marker present) and leaves a whole bag
(metadata.yaml, chunk index, converts without --recover). 54 / 54.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…-303 adoption week, item 2)

Decision note docs/typed-distribution-decision.md: per-PC MSVC build (35 s + v143 +
detached-environments + an unmerged upstream fix on every PC) vs a CI-built conda
channel (5–9 s, no compiler, one sha256-pinned build so rmw_zenoh type hashes agree).
Recommendation: the channel; host = the in-repo channel/ read over raw.githubusercontent.com
at an arena-interfaces-v* TAG (T6 measured), moving to a prefix.dev lab channel when the
lab owns one. The per-PC build stays the .msg author's path.

Repo side: .github/workflows/arena-interfaces.yml builds win-64 / osx-arm64 / linux-64
(windows-2022 → v143; detached-environments C:\px for MAX_PATH; pixi v0.80.0 pinned
because the backend is not in the lock), indexes the three with rattler-index and
publishes the `arena-interfaces-channel` artifact; on a tag it also uploads to prefix.dev
when ARENA_INTERFACES_CHANNEL / PREFIX_API_KEY are set. Build-only on PRs touching the
package. The release recipe (version bump, download, tag, pixi.toml edits) is in the note.
pixi.toml is unchanged: the default String-only environment is untouched. Verified
locally on the Mac: pixi build 17 s, rattler-index fs → repodata; the Windows runner is
verified by the PR run this push triggers.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ad of GitHub (LAB-303 S1/S2, item 3)

S1 (ww18 5.7: the bridge was hard-killed mid closed loop, the Studio showed only
"bridge disconnected", the run carried on and BIAS kept recording):
- fictrac-bridge-client emits 'disconnect' {involuntary, url}; involuntary = not our
  disconnect(). Emitted before pending connector waiters are rejected, so the loss
  is recorded as the reason, not a generic send failure.
- ArenaRunner.bridgeInUse: the active run depends on the bridge when a closed loop
  is armed (startClosedLoop), connector plugins are declared, or the bridge is
  recording the run. ArenaSession faults such a run with 'bridge_lost' on an
  involuntary close; a user Disconnect or an unused bridge is not a fault.
- run-log deriveOutcome → BRIDGE_LOST (the adapter now defers to it for faulted
  aborts); transcript line carries the note. Studio: banner + log line, STOP via
  the runner's finally, NO controller post-mortem (the controller is fine). The
  bridge's own record is recovered by its first export after a restart (item 1).

S2 (ww18 5: a tab opened at ?p=… showed no protocol until refreshed): initFromUrl
awaited the course repo's shared-pattern listing (a GitHub API call with no
timeout) BEFORE the ?p= load; on a slow or walled network that held the whole
init and a refresh merely got a warm connection. Now the listing starts first
and gets ≤ 1.5 s before the load goes ahead, every pre-step is guarded, and an
unknown protocol key banners instead of leaving an empty Run view.

Footer v0.97 + ET stamp, release notes. Tests: deriveOutcome, adapter outcome,
session bridge-lost hook (4 cases), runner bridgeInUse (unit + closed-loop flow).
Root suite 69/69. Prettier on the changed JS only.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…g before the first acquisition (ScanImage 2026.0.0); ping/status pass on the Bergamo PC
…ws even after grabs on ScanImage 2026.0.0); laser-clocked vDAQ noted; single connector grab passes
…aded server stalled the Studio's module imports in Chrome; localhost cost ~2 s/connection on Windows)
…z on the Bergamo, not ~29); dry-run results logged
…gamo card's slots (violet 7, blue 3, green 5, yellow 9)
… colors 2-4, 640 frames cut the end of duty 255); run 1 logged
…ottling (Studio window hidden/occluded); workaround logged
…uty steps visible in both channels, per color)
… x14) + frame-continuity check of runs 1-2 (0 gaps, 29.12 Hz)
mbreiser and others added 3 commits October 18, 2026 00:59
…3 S1)

- bridgeRequired only for a recorded run the bridge is actually logging
  (bridge.logging), not merely connected for the scope.
- A log_control_ack slower than the 800 ms wait still names the run's bridge
  file: the run subscribes to 'loglevel' once and takes `file` for THIS run.
- disconnect() and a connect() over a still-closing socket settle every
  pending request (export, level ack) themselves; the superseded socket's
  late onclose is ignored, so it must not be the only place they settle.
- The bridge-lost alert no longer hardcodes `pixi run bridge` (wrong on the
  ROS 2 graph): "restart the bridge program on the rig PC".

Tests: client 187 (two new reconnect cases), studio-shell 81 (wiring checks).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… this branch's own S1/S2

#257's bridge-loss model replaces 484b187's: the client emits 'lost' only for an
OPEN socket closing unasked (failed connects and superseded sockets are not
losses), fails every pending request first, and the RUNNER decides via
onBridgeLost (apply on / bridge-routed steps / bridgeRequired). The runner's
bridgeInUse getter and the 'disconnect'-involuntary path are removed; so is the
fault('bridge_lost') outcome path (BRIDGE_LOST now comes from summary.bridgeLost).

Connector delta on top of #257 (its PR body's "item 3"):
- _failPending settles connector_command waiters (outcome 'unknown');
- sequenceUsesBridge counts connector plugin instances (BiasPlugin, ScanImagePlugin)
  as bridge-routed steps, so a loss in an open-loop ITI before a camera step stops the run;
- the bridge-lost alert says devices driven through the bridge were not told to stop.

Also: the run-end bag close in onRunFinished referenced an undeclared `bridge`
(a ReferenceError swallowed by its catch — the fix was a no-op); it now uses
Studio.session.bridge, closes before the upload, and only for this run's log.
initFromUrl takes #257's fire-and-forget listings over the grace-period race;
patByteSource keeps rc5's background-lane version with the layered preview map.

Tests: root suite 70/70 files (runner 544, client 199, session 106, adapter 34,
run-log 39, studio-shell 82, preview-sources 23).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…viz port

Studio / runner / client
- connectorCommand waits timeout_s + CONNECTOR_TIMEOUT_MARGIN_MS (default 15 s): the
  browser outlasts the device node's own deadline, so a reply at the deadline is an
  outcome, not an 'unknown'. 2P protocol's waitIdle already declares timeout_s.
- plugin-registry: ScanImagePlugin `set.value` is text (the allowlist holds numbers,
  booleans and paths); the node coerces "640" / "true" to the JSON scalar.
- protocol-yaml-v3: `variables:` written as a list of anchored scalars (`- &name v`,
  the 2P protocols) kept their identity wrong — `.anchor` was read off a primitive
  (String.prototype.anchor), hence the 2026-10-16 "Variable "function anchor() …"
  is defined more than once" warning. extractVariables/_findVariablePair handle
  both shapes; v3 round-trip suite 39 (823/823).
- runner test: connector plugin steps make a run bridge-dependent (loss in the ITI
  before a camera step → BRIDGE_LOST; without the map → continues). 547/547.
- footer stamp; release notes v0.97 extended; CLAUDE.md: ROS-era rules.

ros2-poc
- mcap_index.reindex replaces the bag atomically (os.replace), never unlink-then-rename.
- recorder: ONE constant storage sidecar per log dir (not one hidden file per run);
  optional max_bag_bytes → bridge_node --max-bag-mb (off by default: bags are archived
  on a share, not pushed to GitHub).
- scanimage_node: coerce_scalar for `set` values.
- kf_row lives once in nodes/_rows.py (no ROS imports), used by the node and the
  converter — `converted == live` depends on one encoder.
- ported from #252: nodes/viz_node.py, foxglove/*.json, tests/test_viz_node.py,
  sizing/; rig.py --foxglove also starts viz_node (+ --viz-args); pixi tasks viz,
  test-viz, fox-json (nodes/foxglove_json.py salvaged from the data branch).
- repo hygiene: colcon build logs (interfaces/log) and the one-off win-64 channel
  payload removed from the tree (+ .gitignore); the typed-prebuilt env still reads
  the pinned commit URL.
- stamp.py: one stamping path again (stamp_tree takes a skip set); tiers 109/109.

Harness: ros2-poc `pixi run test` all PASS (default env), test-viz PASS.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@mbreiser mbreiser changed the title LAB-303 Plan B spike: ROS Studio — MCAP record of truth, typed interfaces, converter (DO NOT MERGE before lab validation) LAB-303: ROS 2 bridge graph + MCAP record, ScanImage/BIAS connectors, Studio v0.97 (candidate 2026-10-20) Oct 18, 2026
…p) + a G6 .pat reader

tools/tiff_run_to_mcap.py: ScanImage TIFFs (frames, channel ids and frame rate from the SI
header tag, per-frame timestamps and the DO1 aux-trigger stamps) + the run's .pat files →
one MCAP with Foxglove well-known JSON schemas: /scanimage/preview/ch<id> (observed, fixed
int16→8-bit window per channel), /scanimage/frame_ref, /scanimage/qc (whole-field mean
minus the pre-stimulus baseline), /arena/stimulus (commanded from a run log, else
RECONSTRUCTED from the first DO1 pulse + nominal step lengths), /arena/display and
/arena/scene (RECONSTRUCTED from .pat × duty × LED color), /run/events and a
/run/provenance message naming the source of every topic. Also writes a Foxglove layout.
tools/patfile.py: minimal G6PT V1/V2 reader mirroring js/pat-parser.js. `pixi run tiff2mcap`.

First use: the 2026-10-16 Bergamo color sweep, run 2 (four 800-frame TIFFs, 29.12 Hz, DO1
spans 24.0 / 24.0 / 24.0 / 23.0 s as the day's analysis found).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
mbreiser added a commit that referenced this pull request Oct 18, 2026
…ader no longer lists manual steps

The 2P color LED-intensity sweep (first run on the Bergamo 2026-10-16, on PR #254's
branch) now points at configs/rigs/bergamo_g6_2x10_2p.yaml, declares
`requires: [controller_block]` and the `controller:` block (triggered, line_sync_safe,
panel firmware "2p"), and its header says that the Studio asserts the four Console
steps it used to ask the operator to do by hand (panel mode 2, DIO 1 role, 300 Hz,
panel firmware). What remains manual: pick the Bergamo session rig, Verify panels
once per session, telemetry on.

This file also exists on #254 (candidate 2026-10-20) with the old header and the cshl
rig; when that lands on main, resolve the add/add conflict in favor of this copy. The
pattern folder and the protocols/index.json entry stay with #254.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@mbreiser
mbreiser marked this pull request as ready for review October 18, 2026 17:57
…em on another Windows PC (LAB-303)

shm_bench.py (the Mac script), shm_bench_qpc.py (the same with perf_counter_ns stamps:
Python 3.12's time.monotonic on Windows ticks every 15.625 ms) and mcap_bench.py (a rosbag2
C++ recorder writing MCAP as the only subscriber, rmw publish -> recorder latency from the
MCAP timestamps). docs/shm-bench.md has the commands, what to check and the ww18 numbers;
WINDOWS-TEST.md step 8 points to it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants