From afbb5cc9aeb878823d452197fbdb7e712e82bbd9 Mon Sep 17 00:00:00 2001 From: Michael B Reiser Date: Thu, 24 Sep 2026 00:15:05 -0400 Subject: [PATCH 1/3] feat(studio): assert sticky controller settings before every run; controller: block, rig defaults, run header (v0.88) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Design: docs/development/controller-settings-strategy.md (Codex-reviewed). Wiring lives in the rig YAML, experiment semantics in the protocol, and every sticky setting a run depends on is asserted (GET → SET → GET) before the run, refused on mismatch, and recorded in the run header. No restore-at-end. - js/controller-settings.js (new): pure planner — normalizeControllerBlock, planAssertions (rig identity, panel mode + capability, refresh derived from pattern grayscale + rig cap, panel firmware footer + fleet verify), formatSnapshot. tests/test-controller-settings.js (42 checks). - js/plugin-registry.js: parseRigIo → defaults.panel_mode, limits.max_refresh_hz, requires.panel_firmware, strict, name. - js/protocol-yaml-v3.js: top-level controller: block (extractControllerBlock; malformed values are blocking errors; missing requires token warns); WEB_RUNNER_CAPABILITIES = [controller_block]. - arena_studio.html: rig defaults.panel_mode applied at connect; Studio.assertControllerSettings runs in runOnce for Run AND Test, blocks with a banner; readControllerSnapshot → run header meta.controller + transcript line; run gate shows the planned verdict; Settings → Controller card with live snapshot and Verify panels (fleet 0xC9); batch flasher stores its verify result. - validate-protocol.mjs: prints the block, cross-checks the rig file (cap, policy, firmware). - configs/rigs: bergamo_g6_2x10_2p.yaml (triggered, 300 Hz cap, firmware 2p, strict) + index; course rigs gain defaults.panel_mode: persistent. - protocols: both 2P protocols point at the Bergamo rig and carry the controller block. - docs: controller-settings.md (user), strategy (implemented), SKILL.md section, rig checklist, release notes v0.88; pixi test list. Tests: full pixi run test green (one legacy assertion updated for the new capability token); every inline Studio script parses. Co-Authored-By: Claude Fable 5.1 --- .agents/skills/protocol-yaml/SKILL.md | 33 ++ .../scripts/validate-protocol.mjs | 40 +++ arena_studio.html | 311 +++++++++++++++++- configs/rigs/bergamo_g6_2x10_2p.yaml | 55 ++++ configs/rigs/cshl_g6_2x10.yaml | 5 + configs/rigs/cshl_g6_2x10_ball.yaml | 5 + configs/rigs/cshl_g6_2x8.yaml | 5 + configs/rigs/g6_3x10.yaml | 5 + configs/rigs/g6_4x10.yaml | 5 + configs/rigs/index.json | 6 + .../development/2p-line-sync-rig-checklist.md | 10 +- .../controller-settings-strategy.md | 106 ++++++ docs/development/controller-settings.md | 124 +++++++ js/controller-settings.js | 272 +++++++++++++++ js/plugin-registry.js | 53 +++ js/protocol-yaml-v3.js | 95 +++++- protocols/g6_2x10_2p_diagnostics.yaml | 15 +- protocols/g6_2x10_2p_duty_sweep.yaml | 18 +- tests/fixtures/v3_controller_block.yaml | 54 +++ tests/test-controller-settings.js | 117 +++++++ tests/test-protocol-roundtrip-v3.js | 41 +++ tests/test-studio-runtime-vars.js | 2 +- 22 files changed, 1367 insertions(+), 10 deletions(-) create mode 100644 configs/rigs/bergamo_g6_2x10_2p.yaml create mode 100644 docs/development/controller-settings-strategy.md create mode 100644 docs/development/controller-settings.md create mode 100644 js/controller-settings.js create mode 100644 tests/fixtures/v3_controller_block.yaml create mode 100644 tests/test-controller-settings.js diff --git a/.agents/skills/protocol-yaml/SKILL.md b/.agents/skills/protocol-yaml/SKILL.md index 04670c18..8324f96b 100644 --- a/.agents/skills/protocol-yaml/SKILL.md +++ b/.agents/skills/protocol-yaml/SKILL.md @@ -314,6 +314,39 @@ changes may tune a trial, never restructure it. Web-only; MATLAB ignores `runtim to run it** until the web runner implements that capability; MATLAB may run it. Leave it out otherwise. +### `controller:` — sticky controller settings the run must assert (Studio v0.93) + +The controller keeps a few **sticky** settings across runs (panel display mode, refresh rate, +Digital IO roles…). Wiring belongs in the rig YAML; what the *experiment* needs goes in a +top-level `controller:` block, and the Studio asserts it before every run (GET → SET → GET, +refuse on mismatch) and writes the before/after snapshot into the run header. Design: +`docs/development/controller-settings-strategy.md`; user doc: `docs/development/controller-settings.md`. + +```yaml +rig: "./configs/rigs/bergamo_g6_2x10_2p.yaml" +requires: [controller_block] # a runner without this feature REFUSES the file instead of ignoring the block +controller: + panel_mode: triggered # 0 oneshot | 1 persistent | 2 triggered | 3 gated (names or numbers) + refresh_policy: line_sync_safe # cap the re-stream rate at the rig's limits.max_refresh_hz — OR refresh_hz: + panel_firmware: "2p" # SD footer version (or prefix) the panel image must match; "2p" = the line-sync build +``` + +- **Most protocols do not need the block.** A course protocol inherits the rig's + `defaults.panel_mode` (persistent); the Studio asserts that too. Only declare a mode when the + experiment depends on it (imaging: `triggered`). +- **A protocol with a `controller:` block is strict**: rig mismatch (its `rig:` vs the session + rig), an unreadable setting, or the wrong panel firmware **blocks the run** (green and blue + buttons). On rigs with `strict: true` in the rig YAML this holds for every protocol. +- `refresh_hz` and `refresh_policy` are mutually exclusive. Prefer the policy: the refresh a + pattern needs follows its grayscale (GS2 streams at 1000 Hz by default, GS16 at 300), so the + Studio derives it from the referenced patterns and the rig cap; a bare number is for refresh + experiments only and must not exceed the rig's `limits.max_refresh_hz`. +- Panel firmware is checked against the SD footer (what image was uploaded) **and** the + session's fleet verify (which panels actually run it). Unverified panels only warn; a failed + verify blocks. Settings → Controller → *Verify panels* runs the fleet check (~1 s per panel). +- The validator prints the block and cross-checks it against the rig file when that is in the repo. +- MATLAB does not implement the block yet; the `requires` token is what makes it refuse. + ## Plugins `plugins:` normally comes from the rig (the Studio pre-fills New protocols from the rig diff --git a/.agents/skills/protocol-yaml/scripts/validate-protocol.mjs b/.agents/skills/protocol-yaml/scripts/validate-protocol.mjs index 7a9ec9f6..effc7bd8 100644 --- a/.agents/skills/protocol-yaml/scripts/validate-protocol.mjs +++ b/.agents/skills/protocol-yaml/scripts/validate-protocol.mjs @@ -71,6 +71,46 @@ if (Array.isArray(exp.requires) && exp.requires.length) { console.log('ℹ requires: [' + exp.requires.join(', ') + ']' + (missing.length ? ' — the WEB runner refuses to run this (missing: ' + missing.join(', ') + ')' : '')); } +// ── controller: block (sticky controller settings asserted before the run) ────── +// docs/development/controller-settings-strategy.md. Malformed values are already in +// `blocking` (collectBlockingErrors); here: what it declares, the capability token, and a +// cross-check against the rig file when it is readable from the repo root. +const cb = exp.controller; +if (cb && cb.declared) { + const parts = []; + if (cb.panel_mode != null) parts.push('panel_mode ' + cb.panel_mode + ' (' + ['oneshot', 'persistent', 'triggered', 'gated'][cb.panel_mode] + ')'); + if (cb.refresh_hz != null) parts.push('refresh_hz ' + cb.refresh_hz); + if (cb.refresh_policy) parts.push('refresh_policy ' + cb.refresh_policy); + if (cb.panel_firmware) parts.push('panel_firmware "' + cb.panel_firmware + '"'); + console.log('ℹ controller: ' + (parts.join(', ') || '(empty block)') + ' — asserted by the Studio before every run, recorded in the run header'); + const rigPath = String(exp.rig_path || ''); + const rigFile = rigPath.startsWith('./') || rigPath.startsWith('configs/') ? resolve(rigPath) : null; + if (rigFile && existsSync(rigFile)) { + try { + const { createRequire } = await import('node:module'); + const Registry = createRequire(import.meta.url)(resolve('js/plugin-registry.js')); + const rig = Registry.parseRigIo(v3.parseRigYAMLText(readFileSync(rigFile, 'utf8'))); + const notes = []; + if (rig.defaults.panel_mode != null && cb.panel_mode != null && rig.defaults.panel_mode !== cb.panel_mode) { + notes.push('protocol panel_mode ' + cb.panel_mode + ' overrides the rig default ' + rig.defaults.panel_mode + ' (allowed; the run asserts the protocol\'s value)'); + } + if (cb.refresh_hz != null && rig.limits.max_refresh_hz != null && cb.refresh_hz > rig.limits.max_refresh_hz) { + console.error('✗ BLOCKING: controller.refresh_hz ' + cb.refresh_hz + ' exceeds the rig limit max_refresh_hz ' + rig.limits.max_refresh_hz + ' (' + rigPath + ')'); + process.exitCode = 1; + } + if (cb.refresh_policy === 'line_sync_safe' && rig.limits.max_refresh_hz == null) { + console.warn('⚠ controller: refresh_policy line_sync_safe but the rig ' + rigPath + ' declares no limits.max_refresh_hz — nothing to cap at'); + } + if (rig.requires.panel_firmware && cb.panel_firmware && rig.requires.panel_firmware !== cb.panel_firmware) { + notes.push('protocol requires panel firmware "' + cb.panel_firmware + '", rig requires "' + rig.requires.panel_firmware + '" — both are checked'); + } + for (const w of rig.warnings || []) console.warn('⚠ rig ' + rigPath + ': ' + w); + for (const n of notes) console.log('ℹ ' + n); + console.log('ℹ rig ' + rigPath + ': defaults.panel_mode ' + rig.defaults.panel_mode + ', limits.max_refresh_hz ' + rig.limits.max_refresh_hz + + ', requires.panel_firmware ' + JSON.stringify(rig.requires.panel_firmware) + ', strict ' + rig.strict); + } catch (e) { console.warn('⚠ rig ' + rigPath + ' could not be cross-checked: ' + (e && e.message ? e.message : e)); } + } +} const rcNames = Object.keys(exp.runtime_controls || {}); if (rcNames.length) { const { createRequire } = await import('node:module'); diff --git a/arena_studio.html b/arena_studio.html index 0cd898fb..6c4aecbf 100644 --- a/arena_studio.html +++ b/arena_studio.html @@ -3360,6 +3360,7 @@

Import error

+ @@ -4151,6 +4152,16 @@

Import error

' matches multiple SD files — delete or rename the extra copy (Console → Patterns)' }; } + // Sticky controller state (panel mode / refresh / panel firmware) — the planned verdict + // from the last dry preflight; recomputed when protocol, rig or controller changed. + const ctlKey = Studio._controllerGateKey ? Studio._controllerGateKey() : ''; + if (Studio.refreshControllerGate && Studio.session && Studio.session.connected && doc && + (!Studio._controllerGate || Studio._controllerGate.key !== ctlKey)) { + Studio.refreshControllerGate(); // async; calls back into updateRunGate + } + const ctlBlocked = (Studio._controllerGate && !Studio._controllerGate.ok) ? Studio._controllerGate.reason + : Studio.rigStrictFault ? 'rig (strict): ' + Studio.rigStrictFault : null; + if (gate.ok && ctlBlocked) gate = { ok: false, reason: ctlBlocked }; const runBtn = $('btnRunExp'); runBtn.disabled = !gate.ok; // Show the experiment length up front (steps + estimated wall-clock) so you @@ -4160,7 +4171,9 @@

Import error

const estTxt = est ? est.steps + ' step' + (est.steps === 1 ? '' : 's') + ' · ~' + fmtEstClock(est.sec) : ''; $('runHint').textContent = gate.ok ? estTxt : (estTxt ? estTxt + ' · ' + gate.reason : gate.reason); const testBtn = $('btnTestExp'); - testBtn.disabled = !(Studio.session && Studio.session.connected && doc); + // A wrong panel mode ruins a TEST run as much as an experiment: the controller gate applies to both. + testBtn.disabled = !(Studio.session && Studio.session.connected && doc) || !!ctlBlocked; + if (ctlBlocked && !gate.ok) $('runHint').textContent = (estTxt ? estTxt + ' · ' : '') + ctlBlocked; // Non-blocking warning: a pattern NAME doesn't match the SD but a // pattern_ID fallback resolves — the run proceeds playing that index, // NOT the named pattern (silent-wrong-pattern footgun). Distinct from @@ -6444,6 +6457,7 @@

Import error

const fi = Wire.decodeFirmwareInfo(await q(Wire.encodeGetFirmwareInfo())); if (fi && fi.version) { $('cFwName').textContent = 'on SD: ' + fi.version; got.push('SD fw «' + fi.version + '»'); } if (got.length) clog('init from controller: ' + got.join(' · '), 'info'); + if (Studio.refreshControllerGate) Studio.refreshControllerGate(); // sticky-state verdict for the run gate } function downloadBlob(bytes, name, mime) { const blob = new Blob([bytes], { type: mime || 'application/octet-stream' }); @@ -7409,6 +7423,7 @@

Import error

}; invalidateRigIo(); renderRigIoMenu(); + Studio._controllerGate = null; // rig changed → the sticky-state verdict must be recomputed updateIspShowUI(); // the batch progress map needs a 20-panel layout if (window.Scope && window.Scope.applyRigBallDiameter) window.Scope.applyRigBallDiameter(); // scope forward-velocity calibration Studio.refreshState(); // device line + mismatch chip @@ -7596,6 +7611,16 @@

Import error

if (roleOf('cIoRoleAi', io.ai.role) === 'in') { clog('rig io: AI "in" not applied — Mode-4 front-end calibration TBD (bench)', 'warn'); } + // Sticky panel display mode default (rig `defaults.panel_mode`): GET → SET → GET, logged. + // On a strict rig a failure here blocks the run gate until it is resolved. + const CSm = window.ControllerSettings; + const wantMode = CSm && io.defaults ? CSm.normalizePanelMode(io.defaults.panel_mode) : null; + Studio.rigStrictFault = null; + if (wantMode !== null) { + const r = await Studio.assertPanelMode(wantMode, { source: 'rig default' }); + if (r.applied) applied.push('panel mode→' + CSm.panelModeName(wantMode)); + if (!r.ok && io.strict) Studio.rigStrictFault = r.message; + } if (applied.length) { clog('rig io defaults applied (' + (Studio.currentRig && Studio.currentRig.name) + '): ' + applied.join(' · '), 'info'); } else if (manual) { @@ -7603,6 +7628,207 @@

Import error

} }; +// ---- controller settings: assert before run, snapshot into the run header ---- +// docs/development/controller-settings-strategy.md. The planner (js/controller-settings.js) +// is pure; this block is the hardware side: GET → SET → GET with logging, the snapshot the +// run header carries, the run-gate reason, and the fleet firmware verify. +Studio.panelVerify = null; // {at, footer, ok, verified, total, mismatched[]} — batch flasher / Verify panels +Studio.controllerSnapshot = null; // last readControllerSnapshot() +Studio._controllerGate = null; // {ok, reason, key} — consulted by updateRunGate +Studio.rigStrictFault = null; // connect-time rig-default failure on a strict rig +async function ctlSend(bytes, timeoutMs) { + try { + return await Studio.session.send(bytes, { timeoutMs: timeoutMs || 2000, silent: true }); + } catch (_) { + return null; + } +} +Studio.assertPanelMode = async function (want, opts) { + const CS = window.ControllerSettings; + const clog = Studio.clog || Studio.rawLog; + const src = (opts && opts.source) || 'protocol'; + const before = Wire.decodePanelDisplayMode(await ctlSend(Wire.encodeGetPanelDisplayMode())); + if (before === want) { + clog('panel mode: ' + CS.panelModeName(want) + ' (' + want + ') already set — ' + src, 'info'); + return { ok: true, applied: false, before, after: before, message: '' }; + } + await ctlSend(Wire.encodeSetPanelDisplayMode(want)); + const after = Wire.decodePanelDisplayMode(await ctlSend(Wire.encodeGetPanelDisplayMode())); + const ok = after === want; + const message = 'panel mode: ' + (before == null ? '?' : CS.panelModeName(before)) + ' → ' + CS.panelModeName(want) + + (ok ? ' (verified — ' + src + ')' : ' FAILED (read back ' + (after == null ? 'nothing' : CS.panelModeName(after)) + ') — ' + src); + clog(message, ok ? 'info' : 'err'); + if ($('cPanelMode') && after != null) $('cPanelMode').value = String(after); + return { ok, applied: ok, before, after, message }; +}; +Studio.assertRefreshHz = async function (want, opts) { + const clog = Studio.clog || Studio.rawLog; + const src = (opts && opts.source) || 'protocol'; + const before = Wire.decodeRefreshRate(await ctlSend(Wire.encodeGetRefreshRate())); + if (before === want) { + clog('refresh: ' + want + ' Hz already set — ' + src, 'info'); + return { ok: true, applied: false, before, after: before, message: '' }; + } + await ctlSend(Wire.encodeSetRefreshRate(want)); + const after = Wire.decodeRefreshRate(await ctlSend(Wire.encodeGetRefreshRate())); + const ok = after === want; + const message = 'refresh: ' + (before == null ? '?' : before + ' Hz') + ' → ' + want + ' Hz' + + (ok ? ' (verified — ' + src + ')' : ' FAILED (read back ' + (after == null ? 'nothing' : after + ' Hz') + ') — ' + src); + clog(message, ok ? 'info' : 'err'); + if ($('cRrHz') && after != null) $('cRrHz').value = String(after); + return { ok, applied: ok, before, after, message }; +}; +// Everything readable about the controller's sticky state, in one object. ~7 round-trips. +Studio.readControllerSnapshot = async function () { + if (!Studio.session || !Studio.session.connected) return null; + await ensureCapabilities(); + const snap = { + at: new Date().toISOString(), + panel_mode: Wire.decodePanelDisplayMode(await ctlSend(Wire.encodeGetPanelDisplayMode())), + refresh_hz: Wire.decodeRefreshRate(await ctlSend(Wire.encodeGetRefreshRate())), + spi_mhz: Wire.decodeSpiClock(await ctlSend(Wire.encodeGetSpiClock())), + ao_mv: Wire.decodeAoVoltage(await ctlSend(Wire.encodeGetAoVoltage())), + dio_roles: hasIoExt() ? Wire.decodeDioRole(await ctlSend(Wire.encodeGetDioRole())) : null, + panel_firmware: Wire.decodeFirmwareInfo(await ctlSend(Wire.encodeGetFirmwareInfo())), + controller_firmware: Studio.firmware || null, + controller_id: Studio.controllerId || null, + capabilities: Array.isArray(Studio.capabilities) ? Studio.capabilities.slice() : [], + verified: Studio.panelVerify + }; + Studio.controllerSnapshot = snap; + return snap; +}; +// Grayscale of every pattern the sequence references, from the SD listing's cached +// pattern info (fillPatternFramesFromCard) or a direct GET_PATTERN_INFO. Best effort. +Studio.referencedPatternInfo = async function (exp) { + const names = new Set(); + for (const c of (exp && exp.conditions) || []) { + for (const cmd of c.commands || []) { + if (cmd && cmd.type === 'controller' && cmd.command_name === 'trialParams' && cmd.pattern) names.add(String(cmd.pattern)); + } + } + const out = []; + const m = Studio.patternSet || null; // the SD listing (patEntry uses the same object) + for (const name of names) { + let gsVal = null; + const p = m && m.patterns ? Object.values(m.patterns).find((x) => x && x.name === name) : null; + if (p && p.preview && (p.preview.gsVal === 1 || p.preview.gsVal === 2)) gsVal = p.preview.gsVal; + else if (p && Number.isFinite(p.index) && Studio.session && Studio.session.connected && !Studio.session.running) { + const info = Wire.decodePatternInfo(await ctlSend(Wire.encodeGetPatternInfo(p.index), 1500)); + if (info) { + gsVal = info.gsVal; + p.preview = Object.assign(p.preview || {}, { gsVal: info.gsVal, frames: info.frameCount }); + } + } + out.push({ name, gsVal }); + } + return out; +}; +function rigNameForProtocol(exp) { + const rp = exp && exp.rig_path ? String(exp.rig_path) : ''; + if (!rp) return null; + const base = rp.split('/').pop().replace(/\.ya?ml$/i, ''); + const entry = (rigIndex || []).find((r) => r && (r.path === rp || String(r.path || '').split('/').pop().replace(/\.ya?ml$/i, '') === base)); + return entry ? entry.name : null; // unknown (MATLAB-side) rig paths are not compared +} +/** + * The run preflight for sticky controller state. dry=true only plans (for the run gate); + * otherwise executes the SETs and returns the header the run log carries. + * @returns {{ok:boolean, header:object, plan:object}} + */ +Studio.assertControllerSettings = async function (exp, opts) { + const CS = window.ControllerSettings; + const clog = Studio.clog || Studio.rawLog; + const dry = !!(opts && opts.dry); + if (!Studio.session || !Studio.session.connected) return { ok: false, header: null, plan: null }; + const rig = (await fetchRigIo()) || parseRigIo(null); + const snap = await Studio.readControllerSnapshot(); + const patterns = await Studio.referencedPatternInfo(exp); + const sessionRig = Studio.currentRig && Studio.currentRig.name; + const plan = CS.planAssertions({ block: exp.controller, rig, snapshot: snap, patterns, protocolRig: rigNameForProtocol(exp), sessionRig }); + const src = exp.controller && exp.controller.declared ? 'protocol controller: block' : 'rig default'; + const header = { + rig: rig.name || sessionRig || null, + strict: plan.strict, + intended: plan.intended, + before: snap, + actions: [], + after: null, + blocking: plan.blocking.slice(), + warnings: plan.warnings.slice() + }; + if (!dry) { + for (const w of plan.warnings) clog('controller preflight: ' + w, 'warn'); + for (const b of plan.blocking) clog('controller preflight: ' + b, 'err'); + if (plan.ok) { + for (const act of plan.actions) { + const r = act.setting === 'panel_mode' + ? await Studio.assertPanelMode(act.to, { source: src }) + : await Studio.assertRefreshHz(act.to, { source: src }); + header.actions.push({ setting: act.setting, from: act.from, to: act.to, ok: r.ok }); + if (!r.ok) header.blocking.push(r.message || (act.setting + ' could not be set')); + } + header.after = plan.actions.length ? await Studio.readControllerSnapshot() : snap; + } + clog('controller: ' + CS.formatSnapshot(header.after || snap), 'info'); + } + const ok = header.blocking.length === 0; + if (!ok && !dry) Studio.showBanner('Run refused — controller preflight: ' + header.blocking.join('; ')); + Studio._controllerGate = { ok, reason: ok ? '' : 'controller preflight: ' + header.blocking[0], key: Studio._controllerGateKey() }; + return { ok, header, plan }; +}; +Studio._controllerGateKey = function () { + const doc = Studio.currentDoc; + return [doc ? doc.filename : '', doc ? doc.sha256 : '', Studio.currentRig ? Studio.currentRig.name : '', Studio.controllerSnapshot ? Studio.controllerSnapshot.at : '', Studio.panelVerify ? Studio.panelVerify.at : ''].join('|'); +}; +// Recompute the run-gate verdict (plan only, no SETs) when the protocol, rig or controller +// changed. Debounced single-flight; calls updateRunGate when done. +let ctlGateBusy = false; +Studio.refreshControllerGate = async function () { + const doc = Studio.currentDoc; + if (!doc || !Studio.session || !Studio.session.connected || Studio.session.running) { + Studio._controllerGate = null; + return; + } + if (ctlGateBusy) return; + ctlGateBusy = true; + try { + await Studio.assertControllerSettings(doc.experiment, { dry: true }); + } catch (e) { + Studio._controllerGate = { ok: false, reason: 'controller preflight failed: ' + (e && e.message ? e.message : e), key: Studio._controllerGateKey() }; + } finally { + ctlGateBusy = false; + } + if (Studio.updateRunGate) Studio.updateRunGate(); + if (Studio.renderSettingsCard) Studio.renderSettingsCard(); +}; +// Fleet firmware verify: CRC every panel's running app flash against the SD footer +// (G6_VERIFY_PANEL 0xC9). Stops the display first. ~1 s per panel. Result feeds the +// snapshot / run header and the gate. +Studio.verifyPanels = async function () { + const clog = Studio.clog || Studio.rawLog; + if (!Studio.session || !Studio.session.connected) return null; + const cfgName = Studio.currentRig && Studio.currentRig.arenaConfig; + const cfg = cfgName ? getConfig(cfgName) : null; + const arena = cfg && cfg.arena; + const total = arena && arena.num_rows && arena.num_cols ? Number(arena.num_rows) * Number(arena.num_cols) : 20; + if (Studio.quiesceDisplay) await Studio.quiesceDisplay(); + const footer = Wire.decodeFirmwareInfo(await ctlSend(Wire.encodeGetFirmwareInfo())); + const res = { at: new Date().toISOString(), footer: footer ? footer.version : null, ok: false, verified: 0, total, mismatched: [] }; + clog('verify panels: checking ' + total + ' panels against SD footer «' + (res.footer || '?') + '»…', 'info'); + for (let p = 1; p <= total; p++) { + const r = await ctlSend(Wire.encodeG6VerifyPanel(p), 10000); + const v = r && Wire.decodeProgramPanelResponse(r); + if (v && v.ok) res.verified++; + else res.mismatched.push(p); + } + res.ok = res.mismatched.length === 0; + Studio.panelVerify = res; + clog('verify panels: ' + res.verified + '/' + total + ' match' + (res.ok ? '' : ' — MISMATCH on panels ' + res.mismatched.join(', ')), res.ok ? 'info' : 'err'); + Studio.refreshControllerGate(); + return res; +}; + // ---- metadata controlled vocabularies ---- async function loadVocab(path, key) { try { @@ -8752,6 +8978,18 @@

Import error

if (sum) sum.textContent = 'flash · verify · batch'; ispLog('isp batch — ' + summary, nBad ? 'err' : 'info'); isp.reportText = buildIspReportText(states, summary); + // Fleet verify result → controller snapshot / run header / gate (controller-settings). + if (verify) { + Studio.panelVerify = { + at: new Date().toISOString(), + footer: Studio.controllerSnapshot && Studio.controllerSnapshot.panel_firmware ? Studio.controllerSnapshot.panel_firmware.version : null, + ok: nBad === 0 && nVerified === panels.length, + verified: nVerified, + total: panels.length, + mismatched: states.filter((s) => s.state === 'mismatch' || s.state === 'skipped').map((s) => s.panel) + }; + if (Studio.refreshControllerGate) Studio.refreshControllerGate(); + } // Batch over — leave the final report map lit (nothing guarded follows; // All off / Stop clears it). await pushIspStatus(states, { final: true }); @@ -9185,6 +9423,14 @@

Import error

if (!doc || !Studio.session || !Studio.session.connected) return null; const exp = doc.experiment; if (refuseUnsupportedRequires(exp)) return null; + // Sticky controller state (panel mode, refresh, panel firmware) — assert before the run, + // refuse on mismatch, and keep the before/after snapshot for the run header. + Studio._controllerHeader = null; + if (Studio.assertControllerSettings) { + const cs = await Studio.assertControllerSettings(exp); + if (!cs.ok) return null; + Studio._controllerHeader = cs.header; + } const { steps } = RunnerLib.flattenStructure(exp, { shuffle: shuffleInPlace }); const conditionsByName = new Map((exp.conditions || []).map((c) => [c.name, c])); // Run-start snapshot of the panel fleet (0xD1 is a pure read, ~10 ms per page): @@ -9726,6 +9972,19 @@

Import error

build: Studio.build }); Studio.activeRunLog = RunLog.createRunLog({ intent: isExperiment ? 'experiment' : 'test', meta }); + // Controller snapshot (intended / before / actions / after) — the run header that makes a + // run interpretable afterwards (docs/development/controller-settings-strategy.md § 5). + if (Studio._controllerHeader) { + Studio.activeRunLog.meta.controller = Studio._controllerHeader; + const CS = window.ControllerSettings; + const h = Studio._controllerHeader; + Studio.activeRunLog.event('log', { + level: h.warnings && h.warnings.length ? 'WARN' : 'INFO', + message: 'controller: ' + (CS ? CS.formatSnapshot(h.after || h.before) : JSON.stringify(h.after || h.before)) + + (h.actions && h.actions.length ? ' · set ' + h.actions.map((a) => a.setting + ' ' + a.from + '→' + a.to + (a.ok ? '' : ' FAILED')).join(', ') : ' · nothing to set') + + (h.warnings && h.warnings.length ? ' · warnings: ' + h.warnings.join('; ') : '') + }); + } // Arena Studio Alt may attach the proposal's narrowly scoped runtime // controls to this immutable YAML/run identity. The original Studio has no // hook and therefore behaves exactly as before. @@ -12317,6 +12576,56 @@

Import error

} root.appendChild(rigBox); + // ── Controller (sticky settings asserted before every run) ───────────── + // docs/development/controller-settings-strategy.md — the protocol's controller: + // block (read-only here; edit the YAML), the rig's defaults, what the controller + // reports now, and the fleet firmware verify. + const ctlBox = el('div', { class: 'settings-section' }); + ctlBox.appendChild(el('h3', {}, 'Controller')); + { + const CS = window.ControllerSettings; + const cb = experiment.controller || { declared: false, errors: [], warnings: [] }; + const parts = []; + if (cb.declared) { + if (cb.panel_mode != null) parts.push('panel mode ' + (CS ? CS.panelModeName(cb.panel_mode) : cb.panel_mode)); + if (cb.refresh_hz != null) parts.push('refresh ' + cb.refresh_hz + ' Hz'); + if (cb.refresh_policy) parts.push('refresh policy ' + cb.refresh_policy); + if (cb.panel_firmware) parts.push('panel firmware "' + cb.panel_firmware + '"'); + } + const ro = el('div', { class: 'set-ro' + (cb.errors && cb.errors.length ? ' set-warn' : cb.declared ? ' set-ok' : '') }, + cb.declared + ? 'protocol declares: ' + (parts.join(' · ') || '(empty controller: block)') + : 'no controller: block — this protocol inherits the rig defaults (panel mode) and is asserted against them'); + ro.setAttribute('data-help', 'Sticky controller settings the Studio asserts before every run (GET → SET → GET) and records in the run header. Declared under controller: in the YAML (panel_mode, refresh_hz | refresh_policy, panel_firmware) — edit the YAML to change them. A protocol with a controller: block also needs requires: [controller_block].'); + ctlBox.appendChild(ro); + for (const e of cb.errors || []) ctlBox.appendChild(el('div', { class: 'set-ro set-warn' }, '✗ ' + e)); + for (const w of cb.warnings || []) ctlBox.appendChild(el('div', { class: 'set-ro set-warn' }, '⚠ ' + w)); + const req = Array.isArray(experiment.requires) ? experiment.requires : []; + if (cb.declared && !req.includes('controller_block')) { + ctlBox.appendChild(el('div', { class: 'set-ro set-warn' }, '⚠ add `requires: [controller_block]` so a runner without this feature refuses instead of ignoring the block')); + } + const snap = window.Studio && window.Studio.controllerSnapshot; + const st = el('div', { class: 'set-ro' }, snap && CS ? 'controller now: ' + CS.formatSnapshot(snap) : 'controller now: not read yet (connect)'); + st.setAttribute('data-help', 'What the controller reports right now: panel display mode, refresh rate, SPI clock, Digital IO roles, analog-out level, the panel-firmware footer on the SD card, and the last fleet verify.'); + ctlBox.appendChild(st); + const gate = window.Studio && window.Studio._controllerGate; + if (gate && !gate.ok) ctlBox.appendChild(el('div', { class: 'set-ro set-warn' }, '✗ ' + gate.reason)); + else if (gate && gate.ok && snap) ctlBox.appendChild(el('div', { class: 'set-ro set-ok' }, '✓ controller preflight would pass')); + const row = el('div', { class: 'set-row' }); + const vb = el('button', { class: 'zone-head-btn', title: 'CRC-check every panel\'s running firmware against the SD footer (G6_VERIFY_PANEL 0xC9). Stops the display first; about 1 s per panel. The result goes into the run header and the run gate.' }, 'Verify panels'); + vb.disabled = !(window.Studio && window.Studio.session && window.Studio.session.connected && window.Studio.verifyPanels); + vb.addEventListener('click', async () => { + vb.disabled = true; + try { await window.Studio.verifyPanels(); } finally { renderSettings(); } + }); + row.appendChild(vb); + const pv = window.Studio && window.Studio.panelVerify; + row.appendChild(el('span', { class: 'set-ro' }, pv ? (pv.ok ? '✓ ' : '✗ ') + pv.verified + '/' + pv.total + ' panels match «' + (pv.footer || '?') + '» (' + pv.at.slice(11, 19) + ')' : 'panels not verified this session')); + ctlBox.appendChild(row); + } + root.appendChild(ctlBox); + window.Studio.renderSettingsCard = renderSettings; // re-render when the gate / snapshot changes + // ── Pattern set ───────────────────────────────────────────────── const psBox = el('div', { class: 'settings-section' }); psBox.appendChild(el('h3', {}, 'Pattern set')); diff --git a/configs/rigs/bergamo_g6_2x10_2p.yaml b/configs/rigs/bergamo_g6_2x10_2p.yaml new file mode 100644 index 00000000..ccd9e242 --- /dev/null +++ b/configs/rigs/bergamo_g6_2x10_2p.yaml @@ -0,0 +1,55 @@ +# Bergamo two-photon rig — G6 2x10 arena, line-synchronised to the resonant scanner. +# +# This file describes the WIRING and the bench's safety limits; the experiment's own +# needs (panel mode, refresh policy) live in the protocol's `controller:` block. Both are +# asserted by Arena Studio before every run and written into the run header +# (docs/development/controller-settings-strategy.md). +# +# Cabling (rig checklist docs/development/2p-line-sync-rig-checklist.md): +# Digital IO 1 (J3) → vDAQ free digital input (D2.1), ScanImage "Aux trigger 1": +# one 5 V pulse per 300 Hz frame transfer, stamped into every TIFF header. +# Digital IO 2 (J4) ← ScanImage line clock (HIGH while pixels are recorded, LOW in the +# turnaround) = the panels' EINT trigger route (J30 shunt installed). +# Analog out (J27) → optional condition marker into a vDAQ AI; idles at 0 V here. + +format_version: "1.0" +name: "Bergamo G6 2x10 (2P line-sync)" +description: "Bergamo two-photon rig — G6 2x10 arena, panels flash only in the line-clock LOW gap" + +arena: "G6_2x10" + +controller: + host: "0.0.0.0" # USB/serial via the browser port picker; host/port unused + port: 62222 + +plugins: {} + +io: + dio: + - port: 1 + role: out_debug_framescan # J3: frame-transfer pulse → ScanImage aux trigger + default: 0 + - port: 2 + role: in_trigger # J4: line clock in. Declared explicitly — `off` would mean "don't touch" + default: 0 + ai: + role: off + ao: + role: programmable + default: 0.0 # idle at 0 V so protocol markers (0.5–3 V) start from a clean baseline + +# Sticky controller settings this bench needs at connect and at every run start. +defaults: + panel_mode: triggered # 2 — LEDs only inside the line-clock gap. Course rigs use persistent (1). + +# Safety limits the Studio enforces on every protocol run here. +limits: + max_refresh_hz: 300 # a GS2 pattern would default to 1000 Hz; every re-streamed frame costs the panels a line edge + +# Installed state a run needs but cannot set. Checked against the SD footer (GET_FIRMWARE_INFO) +# and the session's panel-verify result before the run; mismatch blocks. +requires: + panel_firmware: "2p" # footer version "2p-[-d]" = the eintlow_2p line-sync variant + +# strict: mismatches (rig identity, unreadable settings) BLOCK the run instead of warning. +strict: true diff --git a/configs/rigs/cshl_g6_2x10.yaml b/configs/rigs/cshl_g6_2x10.yaml index 13aef345..b19b445b 100644 --- a/configs/rigs/cshl_g6_2x10.yaml +++ b/configs/rigs/cshl_g6_2x10.yaml @@ -54,3 +54,8 @@ io: ao: role: programmable # off | programmable (0xA0) | frame_number (bench-pending) default: 5.0 # volts at connect — this hardware idles at 5 V + +# Sticky controller default, asserted at connect and before every run (docs/development/controller-settings-strategy.md). +# A protocol may override it with its own controller: block; a 2P (line-sync) protocol refuses to run here. +defaults: + panel_mode: persistent # 1 — free-running panels; this bench has no line clock diff --git a/configs/rigs/cshl_g6_2x10_ball.yaml b/configs/rigs/cshl_g6_2x10_ball.yaml index f3f6540a..656e6313 100644 --- a/configs/rigs/cshl_g6_2x10_ball.yaml +++ b/configs/rigs/cshl_g6_2x10_ball.yaml @@ -56,3 +56,8 @@ io: ao: role: programmable # driven to `default` at connect (SET_AO_VOLTAGE 0xA0) default: 5.0 # volts — this hardware idles at 5 V + +# Sticky controller default, asserted at connect and before every run (docs/development/controller-settings-strategy.md). +# A protocol may override it with its own controller: block; a 2P (line-sync) protocol refuses to run here. +defaults: + panel_mode: persistent # 1 — free-running panels; this bench has no line clock diff --git a/configs/rigs/cshl_g6_2x8.yaml b/configs/rigs/cshl_g6_2x8.yaml index 626ac6ff..19758cf0 100644 --- a/configs/rigs/cshl_g6_2x8.yaml +++ b/configs/rigs/cshl_g6_2x8.yaml @@ -42,3 +42,8 @@ io: ao: role: programmable # off | programmable (0xA0) | frame_number (bench-pending) default: 5.0 # volts at connect — this hardware idles at 5 V + +# Sticky controller default, asserted at connect and before every run (docs/development/controller-settings-strategy.md). +# A protocol may override it with its own controller: block; a 2P (line-sync) protocol refuses to run here. +defaults: + panel_mode: persistent # 1 — free-running panels; this bench has no line clock diff --git a/configs/rigs/g6_3x10.yaml b/configs/rigs/g6_3x10.yaml index 0de2d3cf..b4bf0bc7 100644 --- a/configs/rigs/g6_3x10.yaml +++ b/configs/rigs/g6_3x10.yaml @@ -31,3 +31,8 @@ io: ao: role: programmable default: 5.0 # volts — this hardware idles at 5 V + +# Sticky controller default, asserted at connect and before every run (docs/development/controller-settings-strategy.md). +# A protocol may override it with its own controller: block; a 2P (line-sync) protocol refuses to run here. +defaults: + panel_mode: persistent # 1 — free-running panels; this bench has no line clock diff --git a/configs/rigs/g6_4x10.yaml b/configs/rigs/g6_4x10.yaml index 7c988963..5ce08995 100644 --- a/configs/rigs/g6_4x10.yaml +++ b/configs/rigs/g6_4x10.yaml @@ -31,3 +31,8 @@ io: ao: role: programmable default: 5.0 # volts — this hardware idles at 5 V + +# Sticky controller default, asserted at connect and before every run (docs/development/controller-settings-strategy.md). +# A protocol may override it with its own controller: block; a 2P (line-sync) protocol refuses to run here. +defaults: + panel_mode: persistent # 1 — free-running panels; this bench has no line clock diff --git a/configs/rigs/index.json b/configs/rigs/index.json index 5a7b8da0..8aee7738 100644 --- a/configs/rigs/index.json +++ b/configs/rigs/index.json @@ -33,6 +33,12 @@ "label": "CSHL G6 — 2×8 of 10 (288°)", "path": "./configs/rigs/cshl_g6_2x8.yaml", "arena": "G6_2x8of10" + }, + { + "name": "bergamo_g6_2x10_2p", + "label": "Bergamo G6 — 2×10 two-photon line-sync (strict)", + "path": "./configs/rigs/bergamo_g6_2x10_2p.yaml", + "arena": "G6_2x10" } ] } diff --git a/docs/development/2p-line-sync-rig-checklist.md b/docs/development/2p-line-sync-rig-checklist.md index 6ddee06c..24342d33 100644 --- a/docs/development/2p-line-sync-rig-checklist.md +++ b/docs/development/2p-line-sync-rig-checklist.md @@ -59,10 +59,12 @@ path are unchanged. | Item | Setting | Wire (Debug ▾ → unlock → Send raw hex) | |---|---|---| -| panel display mode | 2 (Triggered) | `02 1B 02` | -| Digital IO 1 (BNC **J3**) | out_debug_framescan — HIGH during every SPI frame transfer | `03 AC 01 03` (read back: `01 AD`) | -| Digital IO 2 (BNC **J4**) | in_trigger (boot default) — the line-clock input | leave | -| refresh rate | 300 Hz (Gray_16 default) | `03 16 2C 01` | +| **session rig** (top bar) | **Bergamo G6 — 2×10 two-photon line-sync (strict)** | — the protocols' `rig:` must match it or the run is refused | +| panel display mode | 2 (Triggered) — **asserted automatically** by the rig default at connect and by the protocols' `controller:` block before every run; the log shows `panel mode: persistent → triggered (verified …)` | manual fallback `02 1B 02` | +| Digital IO 1 (BNC **J3**) | out_debug_framescan — HIGH during every SPI frame transfer (applied from the rig at connect) | read back: `01 AD` | +| Digital IO 2 (BNC **J4**) | in_trigger — the line-clock input (applied from the rig at connect) | read back: `01 AD` | +| refresh rate | ≤ 300 Hz — **asserted automatically** (rig `limits.max_refresh_hz` + protocol `refresh_policy: line_sync_safe`) | manual fallback `03 16 2C 01` | +| panel firmware | footer `2p-…` **checked automatically**; run Settings → Controller → **Verify panels** once per session (~20 s) so the run header carries a fleet verify | — | | telemetry ring | on (default) | — | | pattern | `frame2_h_ccw_200f`, SD index 4 (built-in g6_2x10 set) — or any **Gray_16** pattern on the card | edit the two anchors at the top of the YAML | diff --git a/docs/development/controller-settings-strategy.md b/docs/development/controller-settings-strategy.md new file mode 100644 index 00000000..5586f4d1 --- /dev/null +++ b/docs/development/controller-settings-strategy.md @@ -0,0 +1,106 @@ +# Controller settings: what lives in the rig config, what lives in the protocol (plan, 2026-09-23) + +Status: **Phase 1 implemented** (Studio v0.88, branch claude/controller-settings-block; user doc +`docs/development/controller-settings.md`). Reviewed adversarially before implementation (Claude proposal → Codex GPT-5.5 +review → reconciled here). Phases 2 (firmware) and 3 (MATLAB) are open. Raw review: `Bergamo_testing/092226_SNR/analysis/codex_review_raw/codex_strategy_review.md`. + +## 1. The problem + +The G6 controller has about a dozen *sticky* session settings (set over the wire, kept until changed or +rebooted). They are controlled by four unrelated mechanisms: firmware boot defaults, the rig YAML `io:` block +applied at connect, protocol commands, and the Studio's locked Controller ▾ menu. The panel display mode, the +single most consequential setting for two-photon imaging, is menu-only: set by hand, not recorded with the +data, not checked before a run (arena firmware issue #60). + +## 2. The rule + +> Wiring goes in the rig config. Experiment semantics go in the protocol. Installed state the protocol needs +> but cannot set goes in `requires:`. **Every sticky setting a run depends on is asserted before the run starts, +> verified after setting, and recorded in the run header.** Cleanup afterwards is best-effort, never the safety +> mechanism. + +The last sentence is the main change from the first draft. "Restore the previous value at the end of the run" +looked tidy but is a trap: a closed browser tab, an unplugged USB cable or a controller reboot skips it, and a +clean 2P run would "restore" persistent mode on a rig still wired for imaging. Assert-before-run makes every +run safe on its own and makes back-to-back protocols independent. + +## 3. Classification + +| setting | sticky | home | mechanism | +|---|---|---|---| +| DIO 1/2 roles, AO mode, AO/DO idle levels | yes | **rig** (wiring) | applied at connect (exists); on the 2P rig DIO 2 must be declared `in_trigger`, not `off` (`off` = don't touch) | +| panel display mode | yes | **rig default + protocol requirement** | rig `defaults.panel_mode` applied at connect (course rigs: 1, 2P: 2); a protocol that *needs* a mode declares `controller.panel_mode`; runner GET → SET → GET before the first condition, refuses on mismatch. Course protocols omit it and inherit the rig default, so a copied 2P protocol cannot damage a course bench silently: it refuses to run there | +| refresh rate | yes | **derived** | not hand-authored. The runner reads the grayscale of every referenced pattern (SD header, 0x88) and picks the firmware default for it, capped by the rig's `limits.max_refresh_hz` (2P: 300). A protocol may declare `controller.refresh_policy: line_sync_safe` to insist on the cap; a bare number is only for explicit refresh experiments | +| per-trial pattern, mode 2/3/4, frame_rate, frame_index, gain, duty | no | **protocol trialParams** | unchanged (frame_rate is already per trial; it is the *refresh* rate that is sticky) | +| AO / DO changes during a run (markers) | yes | **protocol commands** | unchanged (`setAnalogOut`, `setDigitalOut`); preflight verifies AO mode is `programmable` when a protocol uses AO markers | +| panel firmware variant | persistent | **protocol `requires`** | `requires.panel_firmware: "eintlow_2p"` matched against the SD footer version (0xE3) **and** the flasher's per-panel verify status (0xC9); footer alone only proves what was uploaded | +| rig identity | – | **protocol `rig:` must match the session rig** | today the 2P protocols point at the course rig file; a mismatch becomes a blocking preflight error | +| pattern files | persistent | protocol preflight by name (exists) | 2P runs additionally log each pattern's SD header (GS mode, frames, default duty) | +| telemetry ring, SPI clock, diag/SD-diag flags | yes | **console only** | protocols never touch them; the run header snapshots what is readable | +| line-clock presence | – | **firmware follow-up** (issue #60: EINT edge counter in health) | until then a manual confirmation step in the 2P preflight | + +## 4. Mechanism: declaration block, not a command + +A `setPanelDisplayMode` *command* inside a condition was the first idea. It fails in exactly the ways that +matter: it is buried where preflight cannot see it, it leaks into the next protocol, and a copied protocol +puts a course bench into triggered mode with no line clock. The declarative block fixes all three: + +```yaml +rig: "./configs/rigs/bergamo_g6_2x10_2p.yaml" +requires: [controller_block] # capability token: a runner without the block refuses the file +controller: + panel_mode: triggered # 0 oneshot | 1 persistent | 2 triggered | 3 gated (names or numbers) + refresh_policy: line_sync_safe # cap at the rig's max_refresh_hz, derived per pattern grayscale + panel_firmware: "2p" # SD footer version/prefix + the session's fleet verify +``` +(As implemented: `requires:` stays the list of capability tokens; the installed-state requirement lives +inside `controller:` and the rig may carry its own `requires.panel_firmware`.) + +Runner behaviour: after connect-time rig apply, before the first condition: GET each declared setting, SET if +different, GET again; log "panel mode 1 → 2 (verified)"; refuse to run if a SET cannot be verified, a +capability is missing, or a `requires` fails. On clean completion and on cancel: best-effort `allOff` and rig +idle outputs. No "restore previous". + +Known gap: the MATLAB runner ignores unknown top-level keys, so a protocol with a `controller:` block looks +safe but is not enforced there. Until MATLAB implements the block, 2P protocols are Studio-only and say so in +their header (and `requires:` with an unknown key must be *blocking* in every runner that understands +`requires` at all). + +## 5. Run header + +Every run log gets a header with intended / observed-before / SET result / observed-after for each declared +setting, plus a snapshot of: protocol hash, rig file and hash, Studio version, controller firmware and +capability bitmap (0xCB/0xC2), panel mode (0x1C), refresh (0x17), SPI clock (0xC6), DIO roles (0xAD), AO +level (0xA1), panel firmware footer (0xE3) and verify status, pattern headers. Post-run / abort state is +appended best-effort. This alone would have made the 22 Sept data interpretable and needs no schema change, +so it is the first thing to build. + +## 6. Ordered plan (smallest set that fixes 2P without creating a course problem) + +Phase 1, webDisplayTools only, no firmware change: +1. **Run header snapshot** in the web runner (§ 5). Files: `js/arena-runner-g6.js`, `js/studio-runlog-adapter.js`. +2. **2P rig config** `configs/rigs/bergamo_g6_2x10_2p.yaml`: DIO 1 `out_debug_framescan`, DIO 2 `in_trigger`, + AO programmable idle 0 V, `defaults.panel_mode: 2`, `limits.max_refresh_hz: 300`, `requires.panel_firmware`. + Course rigs gain `defaults.panel_mode: 1`. Entry in `configs/rigs/index.json`. +3. **Rig parser + connect-time apply** for `defaults.panel_mode` (GET/SET/GET, logged; blocking when the rig + sets `strict: true`). Files: `js/plugin-registry.js` (parseRigIo), `arena_studio.html` (applyRigIo). +4. **Protocol `controller:` block**: parser + validator + editor "Controller" card + runner preflight + (§ 4). Files: `js/protocol-yaml-v3.js`, `.claude/skills/protocol-yaml/bin/validate-protocol.mjs`, + `js/plugin-registry.js`, `js/arena-runner-g6.js`, `arena_studio.html`; tests in `tests/`. +5. **`requires:` extension** for `panel_firmware` and rig identity; preflight blocks on mismatch. +6. **Refresh derivation** from pattern grayscale + rig cap (`refresh_policy`). +7. **Point both 2P protocols** at the new rig and add the block; update the rig checklist so "set panel mode 2 + by hand" becomes "connect and read the log line"; document Studio-only. + +Phase 2, arena firmware (issue #60): per-trial `panel_mode` in TRIAL_PARAMS; mode in GET_HEALTH and telemetry; +EINT edge counter → preflight "line clock present". + +Phase 3: MATLAB runner support for `controller:` and `requires:`. + +## 7. What the review changed (for the record) + +Codex agreed with the rule and with the declaration block, and changed four things: dropped restore-at-end in +favour of assert-before-run; made refresh derived rather than authored; kept course protocols silent on panel +mode (rig default) instead of having every protocol declare it; and added rig identity, per-panel firmware +verification, AO-mode verification and the MATLAB gap to the inventory. Claude's items that survived +unchanged: the rule itself, the run header, `requires:` for installed state, console-only for bench knobs. diff --git a/docs/development/controller-settings.md b/docs/development/controller-settings.md new file mode 100644 index 00000000..b116cfbc --- /dev/null +++ b/docs/development/controller-settings.md @@ -0,0 +1,124 @@ +# Controller settings: rig defaults, the protocol `controller:` block, the run header + +*User documentation for Studio v0.88+. Design and history: `controller-settings-strategy.md`.* + +The G6 controller keeps a handful of **sticky** settings across runs: the panel display mode +(0 oneshot / 1 persistent / 2 triggered / 3 gated), the re-transmit ("refresh") rate, the Digital IO +roles, the analog-out level, the SPI clock. Before v0.88 they were set by hand in the Controller ▾ +menu and nothing recorded them with the data. Now: + +1. the **rig YAML** declares the bench's wiring and defaults, +2. a **protocol** declares what the experiment needs in a `controller:` block, +3. Arena Studio **asserts** both before every run (read → set if needed → read back), **refuses** to + run on a mismatch it cannot fix, and writes a **controller snapshot** into the run header. + +## 1. Rig YAML + +```yaml +io: + dio: + - port: 1 + role: out_debug_framescan # J3 + - port: 2 + role: in_trigger # J4 — declare it; `off` means "don't touch" + ao: + role: programmable + default: 0.0 +defaults: + panel_mode: triggered # asserted at connect and at every run start (course rigs: persistent) +limits: + max_refresh_hz: 300 # protocols may not exceed this; line_sync_safe pins to it +requires: + panel_firmware: "2p" # the SD footer must match (prefix / dash-token match) +strict: true # mismatches BLOCK instead of warn (default false) +``` + +`io:` is applied at connect as before (needs io_ext firmware). `defaults.panel_mode` is applied at +connect the same way, GET → SET → GET, and logged (`panel mode: persistent → triggered (verified — +rig default)`). On a strict rig a failure there disables the run buttons until it is resolved. + +Built-in rigs: `bergamo_g6_2x10_2p` (triggered, 300 Hz cap, "2p" firmware, strict); all course rigs +(`cshl_*`, `g6_3x10`, `g6_4x10`) now carry `defaults.panel_mode: persistent`, so a bench left in +triggered mode by a previous session is put back before the next run. + +## 2. Protocol `controller:` block + +```yaml +rig: "./configs/rigs/bergamo_g6_2x10_2p.yaml" +requires: [controller_block] +controller: + panel_mode: triggered + refresh_policy: line_sync_safe # or refresh_hz: 300 (not both) + panel_firmware: "2p" +``` + +| key | meaning | if absent | +|---|---|---| +| `panel_mode` | the mode the experiment needs; overrides the rig default | the rig default is asserted | +| `refresh_policy: line_sync_safe` | pin the refresh at the rig's `limits.max_refresh_hz` when the current rate or the referenced patterns' default (GS2 → 1000 Hz, GS16 → 300 Hz) would exceed it | with a rig cap the same derivation runs anyway; without one nothing is touched | +| `refresh_hz` | an explicit rate (refresh experiments); must not exceed the rig cap | — | +| `panel_firmware` | SD footer version or prefix the panel image must match; also checked against the session's fleet verify | the rig's `requires.panel_firmware`, if any | + +Declaring a block makes the protocol **strict**: its `rig:` must be the session rig, every setting it +depends on must be readable, and the panel firmware must match. `requires: [controller_block]` is +what makes a runner *without* this feature (MATLAB today) refuse the file rather than run it in the +wrong mode; the validator and the Settings → Controller card warn when it is missing. + +Most protocols need no block. Course protocols inherit `persistent` from their rig. + +## 3. What happens at run time + +Green **Run experiment** and blue **Test experiment** both go through the preflight: + +1. Read the controller: panel mode (0x1C), refresh (0x17), SPI clock (0xC6), DIO roles (0xAD), AO + level (0xA1), panel-firmware footer (0xE3), controller firmware and capabilities, the last fleet + verify. Read the grayscale of every referenced pattern (0x88). +2. Plan (`js/controller-settings.js` `planAssertions`): rig identity, panel mode (needs the + `v3_triggered` / `v3_gated` capability for modes 2/3), refresh, panel firmware. +3. Execute the SETs, each followed by a read-back. Every step is a line in the Studio log: + `panel mode: persistent → triggered (verified — protocol controller: block)`. +4. If anything blocks: banner *Run refused — controller preflight: …*, nothing runs. +5. The run header (`meta.controller` in the run log JSON, one `controller:` line in the .txt) + carries `intended`, `before`, `actions` (from → to, ok), `after`, `warnings`, `blocking`, `strict`, + `rig`. `before`/`after` are full snapshots (mode, refresh, SPI, DIO roles, AO, footer, controller + firmware/id/capabilities, fleet verify). + +The run gate shows the planned verdict before you press anything (the hint under the run +buttons and Settings → Controller). It is recomputed when the protocol, the session rig or the +controller changes. After the run, the runner's best-effort STOP/allOff is unchanged; there is no +"restore previous mode" — every run asserts what it needs. + +## 4. Panel firmware verification + +Three layers, from cheap to conclusive: + +| when | what | logged where | +|---|---|---| +| every connect and run | **SD footer** (0xE3): which image was last uploaded to the controller (`2p-9014b5bb-d`, `panel-fw-v1.2.0`, …) | Studio log at connect; run header `before.panel_firmware` | +| after a batch flash with *verify* on, or **Settings → Controller → Verify panels** | **fleet verify** (0xC9 per panel): CRC of each panel's running app flash vs the footer; ~1 s per panel, display stopped | Studio log; `Studio.panelVerify`; run header `before.verified` `{at, footer, ok, verified, total, mismatched[]}` | +| planned (firmware) | per-panel version report at boot, host-readable | — | + +Rules: a required firmware that does not match the footer **blocks**; a failed fleet verify +**blocks** and names the panels; "not verified this session" **warns** (strongly worded on strict +rigs). Verification is per session because the answer can change only when someone flashes, and +a flash goes through the same Studio, which records the result. + +## 5. Staging + +- **Phase 1 (this release)**: everything above. Studio-only; MATLAB refuses protocols that carry the + `controller_block` token. +- **Phase 2 (arena firmware, issue reiserlab/LED-Display_G6_Firmware_Arena#60)**: per-trial + `panel_mode` in TRIAL_PARAMS; mode in GET_HEALTH / telemetry; an EINT edge counter so the + preflight can also check "line clock present"; a boot-time per-panel firmware report so the fleet + check needs no display stop (coordinated with the panel boot-up work). +- **Phase 3 (MATLAB runner)**: implement `controller:` and `requires:`; until then 2P protocols are + Studio-only. + +## 6. Files + +`js/controller-settings.js` (planner, tested in `tests/test-controller-settings.js`), +`js/plugin-registry.js` (`parseRigIo` → defaults/limits/requires/strict), `js/protocol-yaml-v3.js` +(`extractControllerBlock`, blocking errors, `controller_block` capability), `arena_studio.html` +(`Studio.assertControllerSettings`, `readControllerSnapshot`, `verifyPanels`, run gate, Settings card), +`.claude/skills/protocol-yaml/bin/validate-protocol.mjs`, `configs/rigs/*.yaml`, +`protocols/g6_2x10_2p_*.yaml`. diff --git a/js/controller-settings.js b/js/controller-settings.js new file mode 100644 index 00000000..e2209cd5 --- /dev/null +++ b/js/controller-settings.js @@ -0,0 +1,272 @@ +/** + * controller-settings.js — the "assert before run" logic for sticky controller settings. + * + * Design: docs/development/controller-settings-strategy.md. A protocol may declare a + * top-level `controller:` block (panel_mode, refresh_hz | refresh_policy, panel_firmware); + * a rig YAML may declare `defaults:` (panel_mode), `limits:` (max_refresh_hz), + * `requires:` (panel_firmware) and `strict:`. Before every run the Studio reads a snapshot + * of the controller, asks planAssertions() what to SET, what to BLOCK on and what to WARN + * about, executes the SETs (GET → SET → GET), and writes intended / before / after into + * the run header. Nothing here touches hardware: the planner is pure so it can be tested. + * + * LOADING: classic