Skip to content

fix: set up one decoder at a time — nine at once stalled SurfaceFlinger (#76) - #127

Closed
compot-dev wants to merge 3 commits into
v2from
fix/surface-teardown
Closed

compot-dev wants to merge 3 commits into
v2from
fix/surface-teardown

Conversation

@compot-dev

Copy link
Copy Markdown
Collaborator

Closes #76 (if the device check confirms it).

Cause (likely): QuadCell.stop() removed the SurfaceHolder callback and released the decoder asynchronously (setEnabled(false) hands the release to a short-lived thread, #32). Right after, stopQuad() hides the grid, so up to nine SurfaceViews lose their surfaces while their MediaCodec instances are still attached and being released — and since the callback was gone, surfaceDestroyed() could not stop rendering first, which Android requires. On the AG-216 this matches what #76 shows: SurfaceFlinger does not take the window's buffers for tens of seconds (Davey! … QueueBufferDuration=48550217000), and after many cycles media.codec hung in D state (load average 74, only a reboot helped). Reproduced today with a script (grid → single → grid): stalls of 26 s, 9 s, 2.7 s and 1 s in 8 cycles.

Fix: a stopped cell keeps its holder callback until surfaceDestroyed(); there it closes the decoder synchronously (it waits for the release in progress — at most one 5 ms poll since #106) and only then removes the callback. surfaceCreated() ignores a stopped cell.

Device check: the same script, 10+ cycles on Aceline-1 — no Davey over 1 s, no ANR, media.codec not in D state.

@compot-dev

Copy link
Copy Markdown
Collaborator Author

/review

@qodo-free-for-open-source-projects

qodo-free-for-open-source-projects Bot commented Oct 13, 2026 •

Copy link
Copy Markdown

Code Review by Qodo

🐞 Bugs (1) 📘 Rule violations (0) 📎 Requirement gaps (1) 🎨 UX issues (0) 🔗 Cross-repo conflicts (0) 📜 Skill insights (0)

Grey Divider


Action required

1. Codec shutdown can race worker exit 📎 Requirement gap ☼ Reliability
Description
QuadCell.stop() starts asynchronous codec release and immediately calls shutdownNow() without
waiting for the decoder worker to terminate. A worker still inside decoding or starting a codec can
overlap with release when the stopped cell's surface is destroyed, reaching the codec lifecycle from
both threads.
Code

android/app/src/main/java/com/openipc/decoder/QuadCell.java[R224-227]

+        // The holder callback stays until surfaceDestroyed(): there the decoder is closed
+        // before the surface goes away. Removing it here let the grid's surfaces be destroyed
+        // while up to nine decoders were still being released on their own threads, and
+        // SurfaceFlinger then stalled the window's buffers for tens of seconds (#76)
Evidence
The retained callback changes the stopped-cell teardown path, while stop() still disables the
decoder before interrupting its executor. The decoder manager releases the codec on a separate
thread, and executor shutdown is not awaited, so codec release is not ordered after worker
termination.

Nonblocking pause and interruptible decoder shutdown
android/app/src/main/java/com/openipc/decoder/QuadCell.java[208-229]
android/app/src/main/java/com/openipc/decoder/MediaCodecManager.java[190-242]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
Stopped cells retain asynchronous codec release while their decode executors are only interrupted, not awaited, so codec operations can overlap with a worker that has not exited.
## Fix Focus Areas
- android/app/src/main/java/com/openipc/decoder/QuadCell.java[208-229]
- android/app/src/main/java/com/openipc/decoder/MediaCodecManager.java[190-242]
## Recommended Fix
Coordinate worker termination and codec release so the codec is released only after the decode worker has exited. Keep `onPause()` and `stop()` nonblocking by completing the wait and release on a worker thread, and ensure a stopped cell cannot create or use another decoder during that transition.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


2. Surface teardown can block the UI ✓ Resolved
Description
The retained SurfaceHolder.Callback calls decoder.closeDecoder() synchronously from
surfaceDestroyed(), where it waits for decoderLock before stopping and releasing an attached
codec. When stopQuad() stops the cells and hides the grid, surface destruction can reach this path
before the asynchronous release worker detaches each codec, making the callback wait for active
decoder work and codec shutdown for each occupied cell.
Code

android/app/src/main/java/com/openipc/decoder/QuadCell.java[R224-227]

+        // The holder callback stays until surfaceDestroyed(): there the decoder is closed
+        // before the surface goes away. Removing it here let the grid's surfaces be destroyed
+        // while up to nine decoders were still being released on their own threads, and
+        // SurfaceFlinger then stalled the window's buffers for tens of seconds (#76)
Evidence
stopQuad() stops every cell before hiding the container, and QuadCell.stop() leaves each holder
callback registered, so hiding the grid can invoke surfaceDestroyed() on stopped cells. That
callback calls closeDecoder() synchronously; when a codec is still attached, closeDecoder()
acquires decoderLock and calls MediaCodec.stop() and release(), while setEnabled(false)
schedules its normal release on another thread.

Nonblocking pause and interruptible decoder shutdown
android/app/src/main/java/com/openipc/decoder/QuadCell.java[158-174]
android/app/src/main/java/com/openipc/decoder/QuadCell.java[224-229]
android/app/src/main/java/com/openipc/decoder/MediaCodecManager.java[213-242]
android/app/src/main/java/com/openipc/decoder/Decoder.java[647-665]
android/app/src/main/java/com/openipc/decoder/QuadCell.java[168-174]
android/app/src/main/java/com/openipc/decoder/QuadCell.java[212-229]
android/app/src/main/java/com/openipc/decoder/MediaCodecManager.java[191-249]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
Stopped cells retain their surface callbacks, allowing grid hiding to trigger synchronous codec shutdown during surface destruction, potentially for multiple occupied cells.
## Fix Focus Areas
- android/app/src/main/java/com/openipc/decoder/QuadCell.java[158-174]
- android/app/src/main/java/com/openipc/decoder/QuadCell.java[212-229]
- android/app/src/main/java/com/openipc/decoder/MediaCodecManager.java[213-242]
- android/app/src/main/java/com/openipc/decoder/Decoder.java[647-665]
## Recommended Fix
Coordinate decoder shutdown and codec release on the decoder worker rather than in the surface callback or a synchronous cell-stop path. Complete the releases before hiding the grid, without making pause or cell-stop callers wait synchronously, and preserve the ordering that prevents rendering into a surface after `surfaceDestroyed()` returns.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


3. Grid surfaces can outlive decoder release 🐞 Bug ≡ Correctness
Description
QuadCell.stop() retains the callback so surfaceDestroyed() can call closeDecoder(), but
setEnabled(false) can detach the decoder under decoderLock and release it after unlocking. If
that release thread detaches first, closeDecoder() sees no decoder and returns while codec release
is still running, allowing grid teardown to proceed.
Code

android/app/src/main/java/com/openipc/decoder/QuadCell.java[R224-227]

+        // The holder callback stays until surfaceDestroyed(): there the decoder is closed
+        // before the surface goes away. Removing it here let the grid's surfaces be destroyed
+        // while up to nine decoders were still being released on their own threads, and
+        // SurfaceFlinger then stalled the window's buffers for tens of seconds (#76)
Evidence
stop() disables decoding and now keeps the callback. The release worker sets decoder to null
before calling releaseNow() outside the lock; the retained callback's closeDecoder() returns
immediately when it sees that null.

android/app/src/main/java/com/openipc/decoder/QuadCell.java[212-229]
android/app/src/main/java/com/openipc/decoder/QuadCell.java[168-174]
android/app/src/main/java/com/openipc/decoder/MediaCodecManager.java[198-220]
android/app/src/main/java/com/openipc/decoder/MediaCodecManager.java[235-249]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
A stopped cell's surface-destruction callback can return while the asynchronous decoder release is still running.
## Fix Focus Areas
- android/app/src/main/java/com/openipc/decoder/QuadCell.java[171-174]
- android/app/src/main/java/com/openipc/decoder/MediaCodecManager.java[198-249]
## Recommended Fix
Track completion of an in-flight decoder release so teardown cannot treat a detached decoder as a completed release. Arrange grid hiding only after outstanding releases finish.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


Grey Divider

Tip of the day
💡 Did you know, you can turn these tips off under Display > Comment parts

More tips ↗ | Customize Qodo ↗ | Qodo docs ↗

Grey Divider

Qodo Logo

@compot-dev

Copy link
Copy Markdown
Collaborator Author

First device run (48e5449, surface fix only): not enough — 10 cycles still gave stalls of 15, 3, 30, 21 and 49 s and an ANR. The longest one (67.8 s in a second run) came on single → grid, i.e. while the grid is built and up to nine hardware decoders are created at once. Second commit: decoders are set up one after another, at least 250 ms apart (a static lock in MediaCodecManager.createDecoder); the surface fix stays (it is the order Android asks for). Device run follows.

@compot-dev

Copy link
Copy Markdown
Collaborator Author

/agentic_review

Comment on lines +224 to +227
// The holder callback stays until surfaceDestroyed(): there the decoder is closed
// before the surface goes away. Removing it here let the grid's surfaces be destroyed
// while up to nine decoders were still being released on their own threads, and
// SurfaceFlinger then stalled the window's buffers for tens of seconds (#76)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Action required

1. Codec shutdown can race worker exit 📎 Requirement gap ☼ Reliability

QuadCell.stop() starts asynchronous codec release and immediately calls shutdownNow() without
waiting for the decoder worker to terminate. A worker still inside decoding or starting a codec can
overlap with release when the stopped cell's surface is destroyed, reaching the codec lifecycle from
both threads.
Agent Prompt
## Issue description
Stopped cells retain asynchronous codec release while their decode executors are only interrupted, not awaited, so codec operations can overlap with a worker that has not exited.

## Fix Focus Areas
- android/app/src/main/java/com/openipc/decoder/QuadCell.java[208-229]
- android/app/src/main/java/com/openipc/decoder/MediaCodecManager.java[190-242]

## Recommended Fix
Coordinate worker termination and codec release so the codec is released only after the decode worker has exited. Keep `onPause()` and `stop()` nonblocking by completing the wait and release on a worker thread, and ensure a stopped cell cannot create or use another decoder during that transition.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

Comment thread android/app/src/main/java/com/openipc/decoder/QuadCell.java Outdated
Comment on lines +224 to +227
// The holder callback stays until surfaceDestroyed(): there the decoder is closed
// before the surface goes away. Removing it here let the grid's surfaces be destroyed
// while up to nine decoders were still being released on their own threads, and
// SurfaceFlinger then stalled the window's buffers for tens of seconds (#76)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Action required

3. Grid surfaces can outlive decoder release 🐞 Bug ≡ Correctness

QuadCell.stop() retains the callback so surfaceDestroyed() can call closeDecoder(), but
setEnabled(false) can detach the decoder under decoderLock and release it after unlocking. If
that release thread detaches first, closeDecoder() sees no decoder and returns while codec release
is still running, allowing grid teardown to proceed.
Agent Prompt
## Issue description
A stopped cell's surface-destruction callback can return while the asynchronous decoder release is still running.
## Fix Focus Areas
- android/app/src/main/java/com/openipc/decoder/QuadCell.java[171-174]
- android/app/src/main/java/com/openipc/decoder/MediaCodecManager.java[198-249]
## Recommended Fix
Track completion of an in-flight decoder release so teardown cannot treat a detached decoder as a completed release. Arrange grid hiding only after outstanding releases finish.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

@qodo-free-for-open-source-projects

Copy link
Copy Markdown

Code review by qodo was updated up to the latest commit 4414500

…ed, before its surface goes away (#76)"

This reverts commit 48e5449.
@compot-dev

Copy link
Copy Markdown
Collaborator Author

Qodo review — decisions: findings 1–3 are all about the first commit (the stopped cell keeping its holder callback). It did not help on the device either, and finding 3 shows why it could not: closeDecoder() returns as soon as the release thread has detached the codec, before the release has finished. Reverted. This PR now only sets decoders up one at a time, 250 ms apart; device run with that alone follows. A proper "release finished before the surface goes" ordering would need release-completion tracking and is not attempted here.

@compot-dev compot-dev changed the title fix: close a grid cell's decoder before its surface is destroyed (#76) fix: set up one decoder at a time — nine at once stalled SurfaceFlinger (#76) Oct 13, 2026
@qodo-free-for-open-source-projects

Copy link
Copy Markdown

Qodo review — decisions: findings 1–3 are all about the first commit (the stopped cell keeping its holder callback). It did not help on the device either, and finding 3 shows why it could not: closeDecoder() returns as soon as the release thread has detached the codec, before the release has finished. Reverted. This PR now only sets decoders up one at a time, 250 ms apart; device run with that alone follows. A proper "release finished before the surface goes" ordering would need release-completion tracking and is not attempted here.

Understood. Findings 1–3 concern the reverted stopped-cell/holder-callback change and do not apply to the current diff. The remaining change serializes decoder setup under CREATE_LOCK and enforces a 250 ms gap between decoder creations; it does not attempt to solve release-completion ordering during surface teardown.

@compot-dev

Copy link
Copy Markdown
Collaborator Author

Device runs on Aceline-1 (10 cycles grid ↔ single each): neither the teardown order (48e5449, reverted) nor one-at-a-time decoder setup (0b87b46) removes the stalls — 0b87b46 still gave Davey! 23 s and 56 s and an ANR within 3 cycles. Closing without merge; findings go to #76.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ANR on AG-216: RenderThread blocked in eglSwapBuffers with 8 multi-view SurfaceViews

1 participant