feat(ipc)!: remove the cache-key mailboxes, control socket stage 5 (for the release after 3.8.1)

[ChuckBuilds]

Merge condition: do not merge yet

This is stage 5 of the control-socket plan in docs/IPC_CONTROL_SOCKET.md. It is a draft on purpose. Merge it only when one of these is true:

• the release that ships feat(ipc): the control socket carries every web command; mailboxes are a fallback (stage 4) #765 (socket-first, mailboxes as fallback) and feat(plugins): request_on_demand() / end_on_demand() -- plugins ask for the screen in-process #768 (BasePlugin.request_on_demand() / end_on_demand()) has been tagged, and one more release cycle has passed after it. Both ship in 3.8.1 (chore: prepare the 3.8.1 release #756, merged), so this waits for 3.8.1's tag plus one more release cycle;
• or the owner decides to merge earlier.

That gives older and third-party plugins a full release to move off display_on_demand_request. The branch is merged with main through 3.8.2 (#776). Its CHANGELOG section sits under ## Unreleased, above ## 3.8.2.

What is removed

Display

• No on-demand mailbox poll: MailboxWatch, MAILBOX_POLL_INTERVAL_WITH_SOCKET / ON_DEMAND_POLL_INTERVAL, _last_on_demand_poll, _mailbox_poll_interval, _consume_on_demand_request and _note_mailbox_request are all gone. _poll_on_demand_requests() now only drains the socket queue and the plugin queue, so it touches no cache key.
• The persisted display_on_demand_processed_id guard is gone. It existed only to stop a mailbox from being replayed after a restart. The in-memory on_demand_request_id check still drops a start that is sent twice.
• The error publisher no longer reads plugin_error_clear_request. Clears arrive only through errors.clear, which clear_now applies.
• Removed from the API: CacheManager.file_signature, src.cache_manager.MailboxWatch, src.ipc.client.should_fall_back (replaced by display_not_listening), src.error_aggregator.ERROR_CLEAR_REQUEST_KEY, and the clear-request plumbing in read_error_report / error_summary_from_report / plugin_health_from_report / request_error_clear (_pending_cutoff, _is_after, applied_clear_cutoff).

Web

• Nothing is written to either mailbox any more.
• POST /display/on-demand/start:
  • No display listening (no_socket / refused, never sent):
    • service stopped and start_service false: 400, as before;
    • otherwise the route starts the service if needed and answers 202 with status: "starting" at once. It never waits for the display.
  • A single background worker in the web process (web_interface/on_demand_dispatch.py) then sends the request every 0.5 s until the display acknowledges it, for up to 45 s after a cold start or 10 s for a running service without a socket yet. Any other failure ends it at once.
  • One start is pending at a time: a newer start supersedes it, and a stop cancels it. The stop then succeeds with cancelled_request_id, even with no display listening. cancel() waits out a send of the old start that is already in flight (at most the 1 s client timeout), so a superseded start never reaches the display after its replacement. CI caught that race; test_a_cancel_waits_for_a_send_in_flight pins it.
  • The outcome shows in the existing routes:
    • GET /display/on-demand/status: state is {status: "starting", source: "web", request_id, plugin_id, ...} while pending, the display's own state once delivered, or {status: "error", error: "start-timeout"} (or the socket's reason) until the display publishes something newer;
    • GET /display/current-status: adds on_demand_pending.
    • A delivered start keeps reading as starting with delivered: true until the display publishes the state that answers it, for at most 30 s (DELIVERED_SHOWN_SECONDS). On ledpi the display acknowledged a start as its socket opened but acted on it ~5 s later, while Vegas built its first strip, so a UI polling every 700 ms flashed idle. The display's on-demand state now names its request_id, and the status routes wait for a match. A display without the field counts any state published after the delivery. Its startup state names no request, so it does not end the wait.
  • Callers:
    • The MQTT bridge already treats any non-error 2xx as success (_call raises only on >= 400 or status: "error"); a new test pins that for 202.
    • The web UI's on-demand modal and "Preview on display" (app.js) treated only status: "success" as taken. They now treat "starting" as taken too: an info toast, and the floating preview opens.
    • The mqtt-notifications plugin does not call the REST route; it uses request_on_demand().
  • Any other failure answers at once: 503 for unknown_command from an older display, disabled/unsupported, busy, timeouts and forbidden; 400 for invalid_args. Every error carries socket_error.
• POST /display/on-demand/stop: 503 when no display is listening and no pending start was cancelled. The message says whether the service is stopped or may still be starting. With stop_service, the route still stops the service and succeeds.
• POST /errors/clear: without the socket it answers 503 with context.socket_error. The message depends on the cause:
  • display not running: the errors shown are from its last run, and the next run starts empty;
  • display too old: restart it;
  • no socket in this process;
  • display failed the clear.
• transport is always "socket", and clear_pending is always false. Both fields stay for API compatibility.

Tests removed (mailbox only): test_on_demand_mailbox.py, plus the mailbox cadence, file_signature, MailboxWatch, deprecation-log and mailbox-fallback classes in test_ipc_stage4.py, test_api_v3_on_demand_socket.py, test_api_v3_on_demand_restart.py and test_error_snapshot_cross_process.py.

What stays, and why

Key / file	Still written by	Still read by
display_current_state	display	GET /display/current-status when the socket cannot answer
display_on_demand_state	display	GET /display/on-demand/status when the socket cannot answer
plugin_runtime_snapshot	display	/plugins/installed (runtime), /plugins/state, the reconciliations, when the socket cannot answer
display-heartbeat.json (tmpfs)	display	/health display_loop, the update health check
display_on_demand_config	display	the display itself: it resumes an on-demand session after a restart (never a mailbox)
plugin_error_snapshot	display	the /errors/* routes (their only source)

"When the socket cannot answer" covers several cases: the display is stopped or still starting, the web user is not yet in the socket's group, the platform has no Unix sockets (Windows), or LEDMATRIX_CONTROL_SOCKET=off. A stopped display in particular still shows its last state through these keys. Retiring them is left for a later change.

A plugin that still writes display_on_demand_request

The display process has no mailbox reader any more, so CacheManager.save_cache (and so set) refuses both retired keys (RETIRED_MAILBOX_KEYS):

• Nothing reaches disk. A file nobody reads would just wear the SD card.
• One warning per writer, per process: Ignored a write to the retired 'display_on_demand_request' cache key by plugin 'on-air': the display no longer reads this file mailbox. Update it to call self.request_on_demand() / self.end_on_demand() instead ...
• How the writer is named: the first frame on the call stack whose self has a string plugin_id and a cache_manager (any BasePlugin). Failing that, the request's own plugin_id, marked (named in the request). Otherwise unknown.

Writers found:

Where	Plugin (version)	Kind
ledmatrix-plugins plugins/birdnet-go/manager.py:455	birdnet-go 1.2.9	fallback only: when request_on_demand is missing, returns None, or raises
ledmatrix-plugins plugins/mqtt-notifications/manager.py:346	mqtt-notifications 1.2.9	fallback only; also when the core refuses the duration (e.g. a string from MQTT JSON)
ledmatrix-plugins plugins/on-air/manager.py:553	on-air 1.2.15	fallback only (start and stop)
ledmatrix-plugins plugins/pomodoro-timer/manager.py:1383	pomodoro-timer 1.3.11	fallback only (start and stop)

No plugin anywhere writes or reads plugin_error_clear_request, and none uses the removed helpers. That includes MailboxWatch, file_signature, should_fall_back, the error-report functions and display_on_demand_processed_id.

Third-party repos in plugins.json (the 8 with their own repo URL) were all cloned and searched, with no hits: BigC07/ledmatrix-f1-live, ant456/ledmatrix-gif-player, sarjent/ledmatrix-golf, ant456/ledmatrix-plex-marquee, ryug0/ledmatrix-dresden-departures, trsadler/MLB-Scoreboard-LEDMatrix, ant456/ledmatrix-sleeper-fantasy, crazzybrad/LEDMATRIX-NASCAR-PLUGIN.

GitHub-wide gh search code hits outside these repos are copies of core, not plugins:

• ant456/ledmatrix-fixes-repo patches/display_controller.py: a patched display controller that reads the mailbox. It will diverge from this change.
• darthdavidthesith/DavidLedMatrixCode: a copy of core.

After this merges, each of the four monorepo plugins can drop its mailbox fallback once its ledmatrix_min_version reaches the release with #768. Until then the fallback only runs where it should do nothing anyway (no display in the process, or a full queue), and on a stage-5 core it costs one warning line.

Behaviour notes for review

• No waiting in a request. A start that has to wait for the display answers 202 at once, and the web process delivers it in the background (see above). A client follows the outcome through the on-demand status route. The pending start lives in the web process's memory, so restarting the web service drops it.
• Windows dev / LEDMATRIX_CONTROL_SOCKET=off. The web UI can no longer start or stop on-demand sessions or clear errors there. The mailbox used to carry those requests.
• Golden traces. The run-loop harness now sends on-demand requests over its fake control socket, the only real path, so four golden traces change:
  • on_demand, on_demand_named_live, on_demand_pinned: starts and stops land at the request instant instead of the next 0.25 s mailbox look, one frame fewer on the screen they end.
  • vegas: the request lands within one frame instead of 263 ms. That moves the later 1 s-throttled WiFi-notice check by under a second.

Tests

• New and changed tests:
  • the retired-key guard: writes are dropped; one warning per (key, writer); the writer comes from the stack, then the payload, then unknown;
  • display_not_listening for every reason, and sent=True;
  • the dispatcher (test/test_on_demand_dispatch.py): acknowledgement, retry then ack, start-timeout, a per-start wait, other failures reported at once, superseded (including an in-flight ack for the superseded start), stop while pending, cancel with nothing pending, outcome lifetime;
  • the routes: 202 answered at once for a cold start and for a running service without a socket, the status routes while pending and after a timeout, a later display state replacing the failure, a stop while pending, a new start superseding a pending one;
  • the MQTT bridge client reading 202 starting as success, and a JS suite (test/js/unit/test_on_demand_starting.js) for app.js;
  • every non-listening reason answered at once with nothing started or written;
  • stop's stopped and starting messages, and stop_service;
  • each errors/clear failure message;
  • the display touching no cache key for socket commands, and a duplicate start applied once.
• Mutation check.
  • 19 mutants on the stage-5 error paths (client.display_not_listening, the start/stop/clear routes, the retired-key guard and its writer naming, the display's duplicate and ownership guards): 19 killed. The one that first survived (dropping the sent check) is now pinned by test_a_request_that_was_sent_had_a_display.
  • 20 more on the background worker, the routes that use it and app.js (no retry, retry everything, no timeout, reporting a superseded result, delivered left pending, no supersede, ignoring a per-start wait, cancel that keeps sending, outcomes that never expire, 200 instead of 202, never submitting, the wrong wait, no supersede and no cancel in the routes, the status routes not showing or always showing the pending start, app.js treating starting as failure): 20 killed. One survived at first (ignoring a per-start wait) and is now pinned by test_a_start_can_carry_its_own_wait. Removing the in-flight wait in cancel() is killed by test_a_cancel_waits_for_a_send_in_flight.
  • 11 more on the delivered-but-not-yet-active state (not reported at all, no cap, cap measured from the wrong time, missing or wrong delivered flag, always shadowing, id match ignored or inverted, timestamp rule inverted, current-status not gated, the display omitting request_id): 11 killed.
• Full suite on Windows, re-run on the final head (merged with origin/main e40bc47, 3.8.2) against a baseline worktree at e40bc47. The baseline has 74 FAILED/ERROR ids. The branch adds nothing and loses nothing, apart from test_config_flows.py::test_multiple_config_changes: it failed once on the branch and fails 2 runs in 6 on the baseline alone (flaky on main). The existing Windows failures are starlark ownership, install_lowmem, pixlet_download and similar. The Linux-only socket tests (TestOverTheSocket, TestRealSocket) skip on Windows and run in CI.
• mypy 1.20.2 (scripts/check_types.py): same result as origin/main. The one existing error (fetch_service.py:676, unused ignore on Windows) is also there on the baseline. cache_manager/error_aggregator/ipc.client drop from 16 to 15 errors (not on the ratchet).
• Rig: not run. ledpi's ~/hwbatch.lock was held by another session (a malloc-trim soak) every time it was checked.

🤖 Generated with Claude Code

[coderabbitai]

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

• Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

• Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[codacy-production]

Up to standards ✅

🟢 Issues 0 issues

Results:
0 new issues

View in Codacy

🟢 Metrics 20 complexity · 0 duplication

Metric	Results
Complexity	20
Duplication	0

View in Codacy

_{NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.}