Skip to content

Multi-view: adaptive decoding — full rate or key frames per cell, learnt device capacity #85

Closed

Description

@compot-dev

Follow-up to #83/#84. Measured on the AG-216: the hardware decoder handles about its declared budget (1080p60 in total), i.e. two 1080p30 cameras. With 8 main streams, #84 pauses 6 cells. Instead, every camera should stay on screen.

Design (agreed):

  1. Key-frame mode: a cell can decode only key frames (IDR + parameter sets) and skip the rest. The picture updates once per GOP (typically 1–2 s), at a small fraction of the load. Corner text: 1920×1080 H.264 key frames · 1 fps.
  2. Start: every cell starts in key-frame mode, which tells its resolution and frame rate within a second or two. Cells are then promoted to full rate in priority order while the estimated load (width × height × fps, in macroblocks/s) fits the budget.
  3. Budget: initially the decoder's declared throughput (getSupportedFrameRatesFor(1920,1080) × 1080p macroblocks); then the learnt capacity of this device, stored in the app settings.
  4. By measurement: decoder losses (queue overflow, no input buffer; see Multi-view: respect decoder limits — instance limit per codec, overload detection, pause cells beyond capacity #83) → demote the lowest-priority full cell to key frames, and record the load at which losses started as the learnt capacity. Stable with headroom (30 s without losses, load < 80 % of the learnt capacity) → try promoting one key-frame cell; if it causes losses, demote it again.
  5. Priority: slot order; the cell selected for audio (long press) first.
  6. Limits that key frames do not lift: decoder instances per codec (AG-216: 4 × H.265). Such cells keep showing Decoder limit: N × H.265. A shared decoder for key frames of several cameras could lift it later (separate issue).

Replaces the pause from #84 for overload; the pause stays only as a last resort, if even key-frame mode loses frames.

Activity

  1. compot-dev commented on Oct 12, 2026

    @compot-dev
    CollaboratorAuthor

    To do tomorrow: Qodo's review of PR #86 (all five accepted)

    • One offline camera keeps all lower cells on key frames: the planner waits for the load of a higher-priority cell that never becomes known. Skip cells that are not connected/playing; wait only, and not indefinitely, for playing cells whose load is still unknown.
    • A full-rate cell can flip between full rate and key frames forever: an ordinary promotion ("fits") ignores the failed load. Apply the failedLoad × 0.9 limit to every promotion, not only to probes.
    • The last working camera can be paused needlessly: the "more than one active cell" check counts cells that are not playing. Count only playing cells, and never pause the last one.
    • Lost key frames can go undetected: losses are checked only at ≥ 5 received fps, and key-frame cells are below that. Detect decoder losses independently of the frame rate.
    • Restarting retries a load that failed: store the failed load with the learnt capacity (settings), so a restart does not repeat the failing probe.

    Then: CI, device test (8 cameras, include one offline camera), merge #86, close #85.

  2. compot-dev commented on Oct 13, 2026

    @compot-dev
    CollaboratorAuthor

    Carried over from #96 (merged in #97), to do in PR #86 after rebasing on v2:

    • /stats.json: per cell, full rate or key frames (and the learnt capacity, if useful).
    • Debug text (D): a line http://<address>:8085/ per device address (NetInfo.addresses()).
  3. added a commit that references this issue on Oct 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions