Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 16 additions & 8 deletions LifeTrac-v25/DESIGN-CONTROLLER/TODO.md
Original file line number Diff line number Diff line change
Expand Up @@ -1972,14 +1972,22 @@ acceptance steps passed — full record in
dual-broker retained-override trap (bench rule: clear retained control
state on BOTH brokers between campaigns), and the deployment rule
`LIFETRAC_TILE_STALE_AFTER_MS ≥ ~1.25 × n_tiles/(SWEEP_STEP × fps)`.
- [ ] **F11 (candidate, from F10 acceptance §5): demote web_ui's seq-gap
keyframe request.** The gap-tolerant apply requests a keyframe on EVERY
base_seq gap — including a single lost delta frame (two such on-air
events in the surgical window, each costing a ~2.4 KB keyframe at the
243 B/frame budget; the 10 s KeyframeRequester throttle is the only
damper). F10's stale reporting makes this largely redundant: staleness
is now detected and repaired tile-by-tile. Same protocol as F10: gate it
behind an env (default unchanged), measure the A/B on air, then flip.
- [x] **F11 DONE + VERIFIED ON AIR 2026-08-02 (branch `f11-kf-on-seq-gap`)
— web_ui's per-gap keyframe request demoted, default OFF.** The
gap-tolerant apply requested a keyframe on EVERY base_seq gap, including
a single lost delta frame; F10's stale reporting made it redundant.
Implementation: structured causes on CanvasUpdate (`seq_gap`,
`tile_error` — no reason-string matching), suppression only for
pure-gap requests under `LIFETRAC_KF_ON_SEQ_GAP=0`; cold start, grid
mismatch, and tile-decode errors always pass through. On-air A/B in
`bench-evidence/F11_kf_on_gap_2026-08-02/RESULTS.md`: gate-off, 4
induced gaps + 2 natural single-frame losses → 6 suppressions, ZERO
keyframe requests, canvas fully healthy (short gaps re-swept below the
stale horizon; gap stragglers repaired by 0x6C in 1–2 report periods);
ungated cold-start request verified live mid-stream. Control = the F10
surgical window (every gap requested, two keyframe trains granted).
Default flipped to 0 after the pass, per the measure-then-flip
protocol; `=1` restores the old behaviour.

### RS-11 — Next-session sequencing, and the one instrument that gates it (added 2026-07-29)

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,13 @@ class CanvasUpdate:
updated_indices: list[int] = field(default_factory=list)
request_keyframe: bool = False
reason: str = ""
# F11: structured causes so the POLICY layer (web_ui) can gate the
# seq-gap keyframe request without parsing reason strings. A gap and a
# tile error can co-occur in one frame; `reason` keeps only whichever
# fired first, so these flags are the reliable discriminator. Canvas
# stays mechanism-only: it always REPORTS, the caller decides.
seq_gap: bool = False
tile_error: bool = False


class Canvas:
Expand Down Expand Up @@ -138,6 +145,7 @@ def apply(self, frame: TileDeltaFrame) -> CanvasUpdate:
# Now we apply the tiles AND request a keyframe in parallel
# so the operator still sees motion immediately.
update.request_keyframe = True
update.seq_gap = True
update.reason = (
f"base_seq gap: got {frame.base_seq}, expected {expected}")
self._last_base_seq = frame.base_seq
Expand All @@ -156,13 +164,15 @@ def apply(self, frame: TileDeltaFrame) -> CanvasUpdate:
LOG.warning("canvas: dropping tile %d codec=%d: %s",
tile.index, frame.codec, exc)
update.request_keyframe = True
update.tile_error = True
update.reason = update.reason or f"codec_decode_error: {exc}"
continue
except Exception as exc:
LOG.exception("canvas: dropping tile %d codec=%d "
"(unexpected transcode failure)",
tile.index, frame.codec)
update.request_keyframe = True
update.tile_error = True
update.reason = update.reason or (
f"tile_apply_error: {type(exc).__name__}: {exc}")
continue
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,175 @@
"""F11 — gating the per-gap keyframe request behind LIFETRAC_KF_ON_SEQ_GAP.

The gap-tolerant canvas (IP-PlanRev) requested a keyframe on EVERY base_seq
gap, including a single lost delta frame. F10's 0x6C stale-tile path now
detects and repairs exactly that damage tile-by-tile, so the per-gap
keyframe is largely redundant. Per the F10 protocol the gate shipped with
the default ON, and the default was flipped OFF after the on-air A/B
passed 2026-08-02 (bench-evidence/F11_kf_on_gap_2026-08-02/RESULTS.md).
These tests pin the gate mechanics:

- Canvas reports structured causes (seq_gap / tile_error) so web_ui's
policy never string-matches reasons.
- web_ui suppresses the publish ONLY for pure-gap requests with the gate
off; cold start, grid mismatch, and tile errors always pass through.
"""

import os
import sys
import unittest
from unittest import mock

sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))

from image_pipeline import codec_decode # noqa: E402
from image_pipeline.canvas import Canvas # noqa: E402
from image_pipeline.frame_format import ( # noqa: E402
CODEC_MONO_G4,
CODEC_WEBP,
TileBlob,
TileDeltaFrame,
)

try:
import paho.mqtt.client # noqa: F401
import fastapi # noqa: F401
except ImportError:
raise unittest.SkipTest("paho-mqtt + fastapi required for web_ui import")

with mock.patch("paho.mqtt.client.Client") as _mqtt_class:
_instance = _mqtt_class.return_value
_instance.connect = mock.MagicMock()
_instance.loop_start = mock.MagicMock()
_instance.subscribe = mock.MagicMock()
_instance.publish = mock.MagicMock()
import importlib
import web_ui
importlib.reload(web_ui) # rebind module-level mqtt stub


def _frame(seq, indices, *, keyframe=False, codec=CODEC_WEBP,
grid=(12, 8, 32)):
gw, gh, px = grid
return TileDeltaFrame(
frame_kind=1 if keyframe else 0, base_seq=seq,
grid_w=gw, grid_h=gh, tile_px=px,
changed_indices=list(indices),
tiles=[TileBlob(index=i, tx=i % gw, ty=i // gw, blob=b"RIFFwebp")
for i in indices],
codec=codec,
)


class CanvasStructuredCauseTests(unittest.TestCase):
def setUp(self) -> None:
self.canvas = Canvas(clock_ms=lambda: 1000)
self.canvas.apply(_frame(0, range(96), keyframe=True))

def test_clean_delta_sets_no_flags(self) -> None:
upd = self.canvas.apply(_frame(1, [3]))
self.assertFalse(upd.request_keyframe)
self.assertFalse(upd.seq_gap)
self.assertFalse(upd.tile_error)

def test_gap_sets_seq_gap_only(self) -> None:
upd = self.canvas.apply(_frame(5, [3])) # expected 1
self.assertTrue(upd.request_keyframe)
self.assertTrue(upd.seq_gap)
self.assertFalse(upd.tile_error)
self.assertEqual(upd.updated_indices, [3], "gap still applies tiles")

def test_tile_error_sets_tile_error_only(self) -> None:
boom = mock.Mock(side_effect=ValueError("bad blob"))
with mock.patch.dict(codec_decode._TRANSCODERS,
{CODEC_MONO_G4: boom}):
upd = self.canvas.apply(_frame(1, [3], codec=CODEC_MONO_G4))
self.assertTrue(upd.request_keyframe)
self.assertFalse(upd.seq_gap)
self.assertTrue(upd.tile_error)

def test_gap_plus_tile_error_sets_both(self) -> None:
boom = mock.Mock(side_effect=ValueError("bad blob"))
with mock.patch.dict(codec_decode._TRANSCODERS,
{CODEC_MONO_G4: boom}):
upd = self.canvas.apply(_frame(7, [3], codec=CODEC_MONO_G4))
self.assertTrue(upd.seq_gap)
self.assertTrue(upd.tile_error)

def test_cold_start_sets_neither_flag(self) -> None:
cold = Canvas(clock_ms=lambda: 0)
upd = cold.apply(_frame(4, [0]))
self.assertTrue(upd.request_keyframe)
self.assertFalse(upd.seq_gap)
self.assertFalse(upd.tile_error)


class WebUiGateTests(unittest.TestCase):
"""Drive web_ui._ingest_tile_delta with the reassembler stubbed to
hand back a fully-parsed frame, and count req_keyframe publishes."""

KF_TOPIC = "lifetrac/v25/cmd/req_keyframe"

def setUp(self) -> None:
self._saved_gate = web_ui._KF_ON_SEQ_GAP
self._saved_canvas = web_ui._image_canvas
web_ui._image_canvas = Canvas(clock_ms=lambda: 1000)
web_ui._image_publisher.canvas = web_ui._image_canvas
web_ui.mqtt_client.publish = mock.MagicMock()

def tearDown(self) -> None:
web_ui._KF_ON_SEQ_GAP = self._saved_gate
web_ui._image_canvas = self._saved_canvas
web_ui._image_publisher.canvas = self._saved_canvas

def _ingest(self, frame) -> None:
with mock.patch.object(web_ui._image_reassembler, "feed",
return_value=frame):
web_ui._ingest_tile_delta(b"\x00")

def _kf_publishes(self):
return [c for c in web_ui.mqtt_client.publish.call_args_list
if c.args and c.args[0] == self.KF_TOPIC]

def test_gate_off_suppresses_pure_gap(self) -> None:
web_ui._KF_ON_SEQ_GAP = False
self._ingest(_frame(0, range(96), keyframe=True))
self._ingest(_frame(5, [3])) # gap: expected 1
self.assertEqual(self._kf_publishes(), [])
self.assertFalse(web_ui._image_publisher.needs_keyframe)
# The gap's own tiles were still applied.
self.assertEqual(web_ui._image_canvas._tiles[3].arrived_ms, 1000)

def test_gate_on_still_requests_on_gap(self) -> None:
web_ui._KF_ON_SEQ_GAP = True
self._ingest(_frame(0, range(96), keyframe=True))
self._ingest(_frame(5, [3]))
pubs = self._kf_publishes()
self.assertEqual(len(pubs), 1)
self.assertIn(b"base_seq gap", pubs[0].args[1])

def test_gate_off_still_requests_on_cold_start(self) -> None:
web_ui._KF_ON_SEQ_GAP = False
self._ingest(_frame(4, [0])) # no keyframe yet
pubs = self._kf_publishes()
self.assertEqual(len(pubs), 1)
self.assertIn(b"before any keyframe", pubs[0].args[1])

def test_gate_off_still_requests_on_gap_with_tile_error(self) -> None:
web_ui._KF_ON_SEQ_GAP = False
self._ingest(_frame(0, range(96), keyframe=True))
boom = mock.Mock(side_effect=ValueError("bad blob"))
with mock.patch.dict(codec_decode._TRANSCODERS,
{CODEC_MONO_G4: boom}):
self._ingest(_frame(7, [3], codec=CODEC_MONO_G4))
self.assertEqual(len(self._kf_publishes()), 1)

def test_default_env_is_gate_off(self) -> None:
self.assertFalse(
self._saved_gate,
"F11 default flipped OFF after the on-air A/B passed "
"2026-08-02 (6 suppressions, zero keyframes, canvas healthy "
"— see bench-evidence/F11_kf_on_gap_2026-08-02)")


if __name__ == "__main__":
unittest.main()
32 changes: 31 additions & 1 deletion LifeTrac-v25/DESIGN-CONTROLLER/base_station/web_ui.py
Original file line number Diff line number Diff line change
Expand Up @@ -1076,6 +1076,25 @@ def _on_mqtt_connect(_c, _u, _flags, rc):
"3.0"))
TILE_STALE_TOPIC = "lifetrac/v25/cmd/tile_stale"

# F11 (2026-08-01, from the F10 acceptance §5 observation): the gap-tolerant
# canvas requests a keyframe on EVERY base_seq gap — including a single lost
# delta frame — and each granted request costs a multi-frame keyframe train
# at the current link budget. F10's stale-tile reporting now detects and
# repairs exactly that damage tile-by-tile at ~zero extra airtime, so the
# per-gap keyframe is largely redundant. Cold start, grid mismatch, and
# tile-decode errors are NOT gated — those are the keyframe request's
# legitimate jobs.
#
# Default flipped to 0 after the on-air A/B PASSED 2026-08-02
# (bench-evidence/F11_kf_on_gap_2026-08-02/RESULTS.md): gate-off, 4
# induced gaps + 2 natural single-frame air losses → 6 suppressions,
# ZERO keyframe requests, canvas fully healthy (F10 repaired the
# stragglers in 1-2 report periods); the ungated cold-start request
# verified live mid-stream. Control (gate-on) is the F10 acceptance
# surgical window: every gap requested, two keyframe trains granted.
# Set LIFETRAC_KF_ON_SEQ_GAP=1 to restore the old behaviour.
_KF_ON_SEQ_GAP = os.environ.get("LIFETRAC_KF_ON_SEQ_GAP", "0") == "1"


def compute_stale_tiles(canvas, now_ms: int, stale_after_ms: int) -> list:
"""Pure: indices of tiles that have not refreshed within the horizon.
Expand Down Expand Up @@ -1151,7 +1170,18 @@ def _ingest_tile_delta(payload: bytes) -> None:
)
_image_publisher.canvas = _image_canvas
update = _image_canvas.apply(frame)
if update.request_keyframe:
# F11: a request caused ONLY by a base_seq gap is suppressed when
# the gate is off — the gap's tiles were still applied (gap-tolerant
# merge) and whatever the lost frames staled is repaired by the 0x6C
# stale-tile path. Any co-occurring tile error un-suppresses.
suppress_gap_kf = (update.request_keyframe and update.seq_gap
and not update.tile_error and not _KF_ON_SEQ_GAP)
if suppress_gap_kf:
# warning, not info: uvicorn leaves the root logger at its
# WARNING default, so an info here is silently dropped — and
# this line is the bench A/B's suppression counter (F11).
logging.warning("kf-on-gap suppressed (F11): %s", update.reason)
if update.request_keyframe and not suppress_gap_kf:
_image_publisher.needs_keyframe = True
_image_publisher.last_keyframe_reason = update.reason
try:
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
# F11 on-air A/B — per-gap keyframe request demoted (2026-08-02)

**SHA under test:** branch `f11-kf-on-seq-gap` (gate commit `7a1dfb1c` on
main @ 02a84f2e). **Verdict: PASS — default flipped to OFF** in the
follow-up commit, per the measure-then-flip protocol (F9 precedent).

## 1. What was measured

web_ui's gap-tolerant canvas requested a keyframe on EVERY base_seq gap,
including a single lost delta frame. F10's 0x6C stale-tile path now detects
and repairs exactly that damage tile-by-tile, so F11 gates the per-gap
request behind `LIFETRAC_KF_ON_SEQ_GAP`. Suppression applies ONLY to
pure-gap requests (structured `seq_gap`/`tile_error` causes on
CanvasUpdate); cold start, grid mismatch, and tile-decode errors always
pass through.

Setup identical to the F10 acceptance (camera feed, DTS profile 2, v3
depth 2, smooth pacing, base-board web_ui with 30 s stale horizon / 3 s
period), gate OFF. All times host/base UTC.

## 2. Control (A side): F10 acceptance surgical window, gate ON

From `bench-evidence/F10_tile_stale_acceptance_2026-08-01/RESULTS.md` §5:
three seq-gap events (one 18-frame resume gap + two natural single-frame
air losses) → three req_keyframe publishes → two 0x60 radiations (10 s
throttle) → keyframe trains granted, each a multi-frame cost at the
243 B/frame budget.

## 3. Treatment (B side): gate OFF, this session

- **Cold start still keyframes (keep-path, on air):** f11_webui restarted
mid-stream 01:44:30 with an empty canvas; "delta arrived before any
keyframe" published 01:44:33, radiated (two 0x60 attempts, converged) —
the ungated path works.
- **Six gap events, six suppressions, ZERO keyframe requests:**
- Gaps 1–3 induced (3 s rx stops at 01:41:03 / 01:41:47 / 01:42:32):
req_keyframe count in the broker tap stayed 0 (pre-log-fix, the
suppression line was swallowed by the root logger's WARNING default —
fixed to logging.warning and re-verified).
- Gap 4 induced (01:45:27): `kf-on-gap suppressed (F11): base_seq gap:
got 205, expected 194` (11 frames).
- Two NATURAL single-frame air losses in the same window — the exact
event class the control paid keyframes for: `got 234, expected 233`
and `got 6, expected 5` — both suppressed.
- **Canvas health under suppression:** short-gap losses re-swept within
the 24 s rotation (below the 30 s stale horizon — no report even
needed); gap-4 stragglers crossed the horizon and F10 repaired them in
1–2 report periods (1–2 tile bitmaps at 01:45:54–01:46:12, then
silence). No lingering staleness, no keyframe.
- **Radiation audit:** all four 0x60 radiations in the session log are
accounted for — two at stream acquisition (01:39:43, rx daemon's own
requester at relaunch) and two for the mid-stream cold start. None
gap-driven.

## 4. Decision

Every keyframe the old behaviour would have paid across six gap events
was avoided with zero canvas harm; the repair burden moved to F10's
existing tile-by-tile path exactly as designed. Default flipped:
`LIFETRAC_KF_ON_SEQ_GAP=0`; set `=1` to restore the old behaviour.

## 5. Raw artifacts

- `tap_base.log` (this directory) — broker tap: tile_stale + req_keyframe.
- Suppression lines quoted inline from `docker logs f11_webui`.
- Untracked working copy at repo root: `_f11_tap_base.log`.
Loading
Loading