Skip to content

fix: single view on a SurfaceView instead of a TextureView (#76) - #185

Merged
compot-dev merged 2 commits into
v2from
fix/surfaceview-76
Oct 15, 2026
Merged

compot-dev merged 2 commits into
v2from
fix/surfaceview-76

Conversation

@compot-dev

@compot-dev compot-dev commented Oct 15, 2026 •

Copy link
Copy Markdown
Collaborator

The single view (video_surface) moves from a TextureView to a SurfaceView, as the grid cells already use (#61).

Why. On the AG-216 TV box (Aceline-1, Android 10, Mali-G31) the TextureView's frames go through the app's own GL composition and window buffer, and that buffer's queueBuffer stalled (Davey!) for seconds, growing with every grid ↔ single switch in one process: up to 14 s within 60 cycles, ANRs before. The experiment (7938365, branch experiment/surfaceview-76) with a SurfaceView removed them: 0 stalls ≥ 2 s in 60 cycles, longest 1.4 s. Measurements: #76 (comment)

The experiment lost three things; this PR puts them back:

  • Pinch-zoom and pan. A SurfaceView has no setTransform(), and setScaleX() / setTranslationX() move its surface only from Android 7 on. The view itself is now laid out zoom × the fitted size, centred, with the pan as left/top margin, which places the surface the same way on every Android version (minSdk 21). Its surface keeps the fitted size (SurfaceHolder.setFixedSize() in fitVideo()), so SurfaceFlinger scales it and an MJPEG frame drawn with lockCanvas() is not up to 5× the screen. Gesture code, clamping (still against the fitted size) and double-tap reset are unchanged; the detectors get the touch in the parent's coordinates, since the view itself moves with the pan (Qodo). One visible difference: zoomed in, the picture now also fills the letterbox bars instead of being cut to the fitted rectangle.
  • Long-press screenshot and debug-page snapshot. No getBitmap() on a SurfaceView: PixelCopy on the existing snapshot thread (new Snapshots.copy(), which the grid cells' copies use too). The UI thread does not wait: the screenshot is saved from the copy callback, on its own thread as before (Cleanup: small code issues #35); the single view's debug snapshot is copied in Snapshots.render() like a grid cell. Snapshots.Cell.frame and recycleFrames() are gone, nothing takes a frame on the UI thread any more. The screenshot is at stream size, at most 3840×2160 (scaled down above, Qodo). Before Android 7: the screenshot shows "Screenshots need Android 7 or newer" (checked before the storage permission), the snapshot cell "no picture".
  • MJPEG → H.264/H.265 in single view. As a grid cell does (feat: MJPEG cells in multi-view (#132) #139): the view is hidden for a new surface, surfaceDestroyed() closes the decoder, drops the surface and shows the view again (unless the grid or U view took the screen meanwhile), and surfacePending ends only in surfaceCreated(). No surface or already hidden: no recreation, the next surface is new anyway. The surface belongs to the SurfaceView and is never released by the app. One holder callback, added once in onCreate().

Z-order: the SurfaceView stays behind the window (no setZOrderOnTop / setZOrderMediaOverlay), so the status line, debug URLs, black clear overlay, stuck banner, the menu popup, dialogs (Reset, URL editor) and WebUI stay above it. It is GONE whenever the grid or the U view is on screen, as before.

To test on a device (Aceline-1 and a phone):

  1. Pinch-zoom and pan in single view: limits at the picture edges, double tap resets; rotate the phone while zoomed (resets).
  2. Long-press screenshot in single view (H.264 and an MJPEG slot): saved PNG at stream size, no UI freeze.
  3. Debug page snapshot (/snapshot.jpg) in single view: the camera's picture in its cell.
  4. An HTTP MJPEG slot → an H.264 slot (and an H.265 slot) in single view, several times: video comes back, no black screen.
  5. The 60-cycle stall test from ANR on AG-216: RenderThread blocked in eglSwapBuffers with 8 multi-view SurfaceViews #76 (grid → slot 4 single 4 s → grid 8 s): Davey! ≥ 2 s count and longest stall against the table in the issue.
  6. Menu, status line, Reset dialog and WebUI over the single view.

Version 2.09.

Fixes #76

The TextureView's frames go through the app's own GL composition and window
buffer; on a Mali-G31 TV box (Android 10) that buffer's queueBuffer stalled
for seconds, growing with every grid/single switch (up to 14 s in 60 cycles).
With a SurfaceView, as the grid cells use, the stalls are gone (#76).

- Zoom and pan: the view is laid out zoom times the fitted size and moved by
  the pan; its surface keeps the fitted size (setFixedSize). Same gestures,
  clamping and double-tap reset as before, on every Android version.
- Long-press screenshot and the debug page snapshot: PixelCopy on the
  snapshot thread (Snapshots.copy), the UI thread does not wait. Before
  Android 7 a screenshot says it is not available, the snapshot cell says
  "no picture".
- MJPEG to H.264/H.265: the view is hidden and shown again for a new surface,
  as a grid cell does (#139); the decoder waits until surfaceCreated() and is
  closed in surfaceDestroyed().
- Version 2.09.
@compot-dev compot-dev added the awaiting device test PR ready, merge only after testing on a real device label Oct 15, 2026
@qodo-free-for-open-source-projects

Copy link
Copy Markdown

PR Summary by Qodo

Move single-video playback to SurfaceView to prevent rendering stalls

🐞 Bug fix ✨ Enhancement ⚙️ Configuration changes 🕐 20-40 Minutes

Grey Divider

AI Description

• Replace single-view TextureView rendering to avoid growing window-buffer stalls on affected TV
 hardware.
• Preserve zoom, screenshots, debug snapshots, and MJPEG-to-H.264/H.265 switching with
 SurfaceView-compatible paths.
• Bump the Android app version to 2.09.
Diagram

graph TD
  A["Video streams"] --> B["Decoder activity"] --> C["Single SurfaceView"] --> D["SurfaceFlinger"]
  C --> E["Snapshots copy"] --> F["Saved PNG"]
  E --> G["Debug JPEG"]
Loading
High-Level Assessment

Use the SurfaceView approach: it matches the existing grid-cell rendering model and addresses the measured TextureView window-buffer stalls. Retaining TextureView and tuning its composition path would leave the reported device-specific bottleneck in place. Device testing should focus on surface recreation, zoom, and capture because automated coverage was not added.

Files changed (5) +135 / -72

Enhancement (1) +27 / -15
Snapshots.javaShare asynchronous PixelCopy across snapshot paths +27/-15

Share asynchronous PixelCopy across snapshot paths

• Adds a PixelCopy helper used by screenshots and grid-cell copies. Single-view debug snapshots now copy from its SurfaceView during rendering instead of receiving a bitmap captured on the UI thread.

android/app/src/main/java/com/openipc/decoder/Snapshots.java

Bug fix (2) +105 / -54
Decoder.javaMove single-view rendering and capture to SurfaceView +104/-53

Move single-view rendering and capture to SurfaceView

• Replaces TextureView callbacks with SurfaceHolder lifecycle handling and recreates the surface when switching from canvas-drawn MJPEG to hardware decoding. Implements zoom through view layout and fitted surface sizing, and moves screenshot capture to asynchronous PixelCopy.

android/app/src/main/java/com/openipc/decoder/Decoder.java

decoder.xmlDeclare the single-video view as a SurfaceView +1/-1

Declare the single-video view as a SurfaceView

• Replaces the video_surface TextureView with a SurfaceView while retaining the existing overlay layout.

android/app/src/main/res/layout/decoder.xml

Other (2) +3 / -3
build.gradleBump Android release to 2.09 +2/-2

Bump Android release to 2.09

• Increments versionCode from 33 to 34 and versionName from 2.08 to 2.09.

android/app/build.gradle

lint-baseline.xmlUpdate touch-accessibility baseline for SurfaceView +1/-1

Update touch-accessibility baseline for SurfaceView

• Changes the existing accessibility warning's view type from TextureView to SurfaceView.

android/app/lint-baseline.xml

@qodo-free-for-open-source-projects

qodo-free-for-open-source-projects Bot commented Oct 15, 2026 •

Copy link
Copy Markdown

Code Review by Qodo

🐞 Bugs (1) 📘 Rule violations (0) 📎 Requirement gaps (0) 🎨 UX issues (0) 🔗 Cross-repo conflicts (0) 📜 Skill insights (0)

Grey Divider


Action required

1. Panning a zoomed picture lags and jitters under the finger ✓ Resolved
Description
applyZoomTransform() now pans by moving the SurfaceView itself (left/top margins on its layout
params). But the GestureDetector that computes the pan step gets its touch events from that same
view, so the step it measures (onScroll's dx/dy) is in coordinates that shift as the view
moves. After each layout pass the finger's position inside the view has shifted by the last pan
step, so the next step comes out too small or zero: the picture falls behind the finger and moves in
stutters on every Android version.
Code

android/app/src/main/java/com/openipc/decoder/Decoder.java[R1881-1884]

+        // With CENTER the left and top margins move the view off the centre
+        params.leftMargin = Math.round(mPanX);
+        params.topMargin = Math.round(mPanY);
+        mSurface.setLayoutParams(params);
Evidence
The touch listener is set on mSurface and passes events to mGestureDetector. Its onScroll adds
-dx/-dy to mPanX/mPanY and calls applyZoomTransform(), which now writes the pan as
leftMargin/topMargin and so moves mSurface on screen. GestureDetector works out dx from
MotionEvent.getX(), which is relative to the view's own position. Before this PR, setTransform()
on the TextureView changed only the drawn content and left the view where it was, so the coordinates
stayed fixed. With the view itself moving by the pan, the same deltas now feed back into the next
measurement.

android/app/src/main/java/com/openipc/decoder/Decoder.java[282-304]
android/app/src/main/java/com/openipc/decoder/Decoder.java[1876-1885]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
Single-view pan now moves the SurfaceView itself through layout margins, but the touch listener (ScaleGestureDetector and GestureDetector) is attached to that same view. Its event coordinates move with the view, so the onScroll deltas feed back on themselves and panning lags and jitters.
## Fix Focus Areas
- android/app/src/main/java/com/openipc/decoder/Decoder.java[300-304]
- android/app/src/main/java/com/openipc/decoder/Decoder.java[1876-1885]
## Recommended Fix
Give the gesture detectors coordinates that do not move with the pan. One option: put the touch listener on the fixed root FrameLayout (`R.id.decoder`) or on a transparent full-screen overlay. The other: keep the listener on `mSurface` but, before passing each event to the detectors, copy it with `MotionEvent.obtain(event)` and `setLocation(event.getRawX(), event.getRawY())` (or offset it by the view's current left/top) so the detectors see screen coordinates.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools



Remediation recommended

2. High-resolution screenshots can crash the app ✓ Resolved
Description
takeScreenshot() allocates an ARGB bitmap at mVideoWidth × mVideoHeight on the UI thread without
bounding the allocation or handling failure. Those dimensions accept any resolution above 64 pixels,
so a sufficiently large stream can exhaust available memory before PixelCopy starts.
Code

android/app/src/main/java/com/openipc/decoder/Decoder.java[1738]

+        Bitmap bitmap = Bitmap.createBitmap(mVideoWidth, mVideoHeight, Bitmap.Config.ARGB_8888);
Evidence
The new allocation uses the stored stream dimensions rather than the fitted view dimensions;
resolution updates impose no upper bound, and the only pre-allocation check tests for positive
dimensions and a valid surface.

android/app/src/main/java/com/openipc/decoder/Decoder.java[1732-1741]
android/app/src/main/java/com/openipc/decoder/Decoder.java[1828-1837]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
A screenshot now allocates a full stream-sized ARGB bitmap on the UI thread, with no allocation limit or failure handling.
## Fix Focus Areas
- android/app/src/main/java/com/openipc/decoder/Decoder.java[1732-1741]
- android/app/src/main/java/com/openipc/decoder/Decoder.java[1828-1837]
## Recommended Fix
Set a safe maximum bitmap size or pixel count, scale oversized requests within that limit, and handle allocation failure by clearing screenshot state and showing an error.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


3. Android 6 users get a needless permission prompt ✓ Resolved
Description
takeScreenshot() requests storage permission before reaching its new Android 7 version check. On
Android 6, a long press can therefore ask for access even though granting it only leads to the
message that screenshots are unsupported.
Code

android/app/src/main/java/com/openipc/decoder/Decoder.java[R1728-1731]

+        if (Build.VERSION.SDK_INT < Build.VERSION_CODES.N) {
+            Toast.makeText(this, "Screenshots need Android 7 or newer", Toast.LENGTH_SHORT).show();
+            return;
+        }
Evidence
The permission branch runs on Android 6 before the newly added version check, while the app still
supports Android versions as low as API 21.

android/app/src/main/java/com/openipc/decoder/Decoder.java[1712-1731]
android/app/build.gradle[23-25]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
Android 6 users are prompted for storage permission for a screenshot feature that now requires Android 7.
## Fix Focus Areas
- android/app/src/main/java/com/openipc/decoder/Decoder.java[1712-1731]
## Recommended Fix
Move the Android 7 support check ahead of the storage-permission request so unsupported devices show the explanatory message without prompting.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


4. Debug snapshots omit the visible zoom 🐞 Bug ≡ Correctness
Description
snapshotCells() now passes mSurface to PixelCopy, which copies its fixed-size surface buffer
without applying the view's layout-based zoom and pan. When someone zooms or pans the single view,
the debug JPEG shows the full unzoomed frame rather than the picture currently visible on screen.
Code

android/app/src/main/java/com/openipc/decoder/Decoder.java[R1410-1411]

+                    // Copied with the grid cells, off the UI thread (#76)
+                    if (c.live) c.view = mSurface;
Evidence
The surface buffer remains at the unzoomed fitted size, while zoom and pan change only the view
layout. The debug path copies that surface and draws the whole resulting bitmap into the cell
without a corresponding crop.

android/app/src/main/java/com/openipc/decoder/Decoder.java[1405-1412]
android/app/src/main/java/com/openipc/decoder/Decoder.java[1853-1857]
android/app/src/main/java/com/openipc/decoder/Decoder.java[1877-1884]
android/app/src/main/java/com/openipc/decoder/Snapshots.java[104-140]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The single-view debug snapshot copies the entire surface buffer even when zoom and pan change the visible viewport.
## Fix Focus Areas
- android/app/src/main/java/com/openipc/decoder/Decoder.java[1405-1412]
- android/app/src/main/java/com/openipc/decoder/Decoder.java[1853-1857]
- android/app/src/main/java/com/openipc/decoder/Decoder.java[1877-1884]
- android/app/src/main/java/com/openipc/decoder/Snapshots.java[104-140]
## Recommended Fix
Carry the single view's zoom and pan into snapshot rendering and crop the copied frame to the visible viewport before drawing its debug cell.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


Grey Divider

Tip of the day
💡 Did you know, you can hide the parts of a finding you never read, like the evidence or the agent prompt

More tips ↗ | Customize Qodo ↗ | Qodo docs ↗

Grey Divider

Qodo Logo

Comment thread android/app/src/main/java/com/openipc/decoder/Decoder.java Outdated
Comment thread android/app/src/main/java/com/openipc/decoder/Decoder.java Outdated
Comment thread android/app/src/main/java/com/openipc/decoder/Decoder.java
Comment thread android/app/src/main/java/com/openipc/decoder/Decoder.java
@compot-dev

Copy link
Copy Markdown
Collaborator Author

Device test 2026-10-15: passed.

Stall test on Aceline-1 (AG-216, .144): fresh app process, then 60 cycles of grid → slot 4 single (4 s) → grid (8 s). Same script and layout as in the #76 comment of 2026-10-15.

build cycles Davey! ≥ 2 s longest ANR
v2.07 (TextureView) 1–30 / 31–60 23 / 17 7 / 10 4.0 s / 14.0 s 0
this PR (2.09, 69dfeff) 1–30 / 31–60 3 / 0 0 / 0 1.5 s / — 0

TECNO KM4 (Android 15), single view on .133:

  • pinch-zoom; pan follows the finger; double-tap resets (checked by hand)
  • long-press screenshot: full 1920×1080 frame saved, UI not blocked
  • debug page snapshot of the single view
  • HTTP MJPEG → RTSP H.264 → MJPEG → H.264 in single view: a picture every time, no black screen, 0 FATAL EXCEPTION / ANR in

@compot-dev compot-dev removed the awaiting device test PR ready, merge only after testing on a real device label Oct 15, 2026
@compot-dev
compot-dev merged commit e0a3778 into v2 Oct 15, 2026
4 checks passed
@compot-dev
compot-dev deleted the fix/surfaceview-76 branch October 15, 2026 12:56
compot-dev pushed a commit that referenced this pull request Oct 15, 2026
compot-dev added a commit that referenced this pull request Oct 15, 2026
fix: single view on a SurfaceView instead of a TextureView (#76)
compot-dev pushed a commit that referenced this pull request Oct 15, 2026
…, debug page, JSON and log; phone menu, single view (SurfaceView, #185), Reset and WebUI
compot-dev pushed a commit that referenced this pull request Oct 15, 2026
The README (EN/RU/ZH) keeps the overview, screenshots, download,
requirements, quick start, a short controls summary, known limitations,
a documentation table, support and license. The details move to
English-only documents in docs/, each with its own table of contents:

- usage.md: menu, gestures and keys, remote, multi-view, Slides, U cell,
  WebUI, share intent, screenshots, Reset
- transports.md: URL formats and options per transport, codecs,
  authentication, what the servers support
- debug.md: debug server, page, snapshot, stream checks, a /stats.json
  field reference from the code, /log, corner text, bug reports
- architecture.md: data path diagram, classes, threads, decode planner
  (#85, #189), SurfaceView rendering (#61, #76), settings, layout
- testing.md: branches, CI, unit tests, versions and releases,
  branch builds, mapping.txt (#181), local builds
- security.md: network, credential store, debug server LAN rule,
  log redaction, WebUI, permissions
- devices.md: verified and not working devices, hardware decoder notes
- toys.md: drones and Wi-Fi toys, FLOW-UFO in testing (#187, #188),
  unknown toys, what to send

The #76 freeze is no longer a known limitation (fixed in 2.09, #185).
The device report form points to docs/devices.md. Docs only, no
version bump.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ANR on AG-216: RenderThread blocked in eglSwapBuffers with 8 multi-view SurfaceViews

2 participants