WIP: Add live device screen streaming to the Screen Capture window v2 by todi1856 · Pull Request #224 · Unity-Technologies/com.unity.mobile.logcat

todi1856 · 2026-09-23T12:51:29Z

Type of change:

Description

The main logcat window is not affected by this change, only screen capture window is affected

How to test? Go to https://yamato.ds.unity3d.com/project/277/branch/livestream/jobDefinition/.yamato%2Fwrench%2Fpackage-pack-jobs.yml%23package_pack_-_mobile_android-logcat/recent-jobs , take latest artifacts, extract them, and add com.unity.mobile.android-logcat-1.5.0.tgz package to Unity. tgz file will have the necessary jar inside.

Video

You can now watch the device's screen live in the Screen Capture window, and click, scroll and type into it from the Editor.

Also, it's not accessible via Window->Analysis->Android Screen Capture

Capturing needs something on the device: External/UnityLogcatServer is a small Gradle project that builds a 17 KB jar, run through app_process as the shell user. It mirrors a display into an ImageReader, JPEG-encodes each frame and writes it over an abstract socket that adb forward exposes. Frames are only produced when the screen changes, so an idle device costs almost nothing.

Input travels back up the same socket: touch, scroll wheel, keys (through the device's own keyboard layout), and Ctrl/Cmd + A, C, V for select-all/copy/paste on the device. Back / Home / Overview buttons sit beside the image, which is the only way in on gesture-navigation devices.

The window was reorganised around it. Screenshots used to overwrite one file in Temp; they're now kept in Library/AndroidLogcat/Screenshots and listed down the left, with the live view as the first row — arrow keys to cycle, F2 rename, Del delete, right-click for Show In Explorer / Open / Save As, Reconnect on the Live row, Ctrl+Shift+S to capture. The window also opens from Window > Analysis > Android Screen Capture. Stream size, JPEG quality and frame rate cap are in Preferences > Analysis > Android Logcat Settings.

Improved screenshot window

Settings

Checklist for PR maker

Have you added a backport label? (new feature, no backport intended — confirm)
Have you updated the Changelog?
Have you added or updated the Documentation to your PR?

Testing status

Existing or new automation tests - what automation was added, changed
New tests are covering
- start/stop live steam
- frame delivery
- touch, keys
- scroll
- restart
- streaming against a device that was asleep
- screenshot list, delete, rename and external-change tests.

Devices:

Pixel 2 (Android 10)
Pixel 9 Pro Fold (Android 16)
- Folding/Unfolding while viewing
CPH1831 (Android 8.1).

Testing checklist

Built and run editor Locally or Yamato
Run on device Android/iOS (if applicable)
All items have tooltips?

A small Java server, built with Gradle into a dex jar and pushed to the device, that mirrors a display into an ImageReader, encodes each frame as JPEG and writes it to an abstract unix socket. It also reads control messages back up the same socket and injects touch, scroll, key and text events. The jar is a build output and is not committed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

The window now lists saved screenshots beside a Live row that mirrors the device. Selecting Live starts the server on the device, shows its frames, and forwards touch, scroll, keyboard and text back to it. Along with it: a shared zoom and pan viewer for the live view and screenshot previews, a details file written beside each capture, a stats column, device rotation, and settings for the stream's size, quality and frame rate. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Integration tests covering the stream end to end: frames arriving, injected touch, scroll, keys and text taking effect on the device, streaming a device that was asleep, and an unknown display id. Edit mode coverage for the details file written beside a screenshot. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

The jar the live stream pushes to the device is a build output and is not committed, so a clone has none. A build_server_jar job produces it and the pack job now depends on it, which is also what puts it in the tarball the test jobs install. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

todi1856 · 2026-09-23T12:51:42Z

/review

u-pr · 2026-09-23T12:51:49Z

ℹ️ Skipped — a harness review for this commit (5aced5d) already ran or is in flight. Same-commit re-runs are paused for 10 min; push a new commit, or re-comment /harness-review in ~10 min to force a fresh one.

_{🤖 Helpful? 👍/👎}

u-pr

💡 Harness Review

The streaming pipeline is thoughtfully integrated, but several reachable UI, persistence, and startup edge cases can leave the displayed or saved content inconsistent or prevent streaming.

_{Reviewed commit 5aced5d}

_{🤖 Helpful? 👍/👎}

todi1856 · 2026-09-23T13:21:02Z

/review

u-pr

💡 Harness Review

The new capture workflow has state-management issues when device selection changes or a live-stream reader does not terminate promptly.

_{Reviewed commit 6508758}

_{🤖 Helpful? 👍/👎}

todi1856 · 2026-09-23T13:36:39Z

/review

u-pr

💡 Harness Review

The new live-view and persistent screenshot workflow has correctness gaps on reachable edge paths.

_{Reviewed commit f1d921f}

_{🤖 Helpful? 👍/👎}

todi1856 · 2026-09-23T13:48:18Z

/review

u-pr

💡 Harness Review

The live-stream feature is broadly cohesive, but keyboard shortcut forwarding can emit an unmatched key-down event for a normal release order.

_{Reviewed commit c58dc0f}

_{🤖 Helpful? 👍/👎}

todi1856 · 2026-09-23T14:29:11Z

/review

u-pr

💡 Harness Review

The streaming and screenshot-management implementation is broadly cohesive, but saving metadata-less screenshots can preserve incorrect provenance.

_{Reviewed commit b5bf917}

_{🤖 Helpful? 👍/👎}

todi1856 · 2026-09-24T08:29:07Z

/design-consistency

…orking, since UseShellExecute changed to false in CoreCLR for default value

todi1856 · 2026-09-29T12:28:29Z

/review

u-pr · 2026-09-29T12:29:16Z

⚠️ Review skipped — this PR is too large to process.

The diff contains approximately 133,820 tokens (~401,461 characters), which exceeds the maximum limit of 120,000 tokens.

Please consider splitting this PR into smaller, focused changes.

_{🤖 Helpful? 👍/👎}

todi1856 and others added 5 commits September 23, 2026 15:40

Document the live stream and update the changelog

5aced5d

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

u-pr Bot reviewed Sep 23, 2026

View reviewed changes

todi1856 added 5 commits September 23, 2026 16:06

fix preview going out of bounds

5db42b1

handle virtualDisplay being null

f87a4dd

Fix save as for screenshot

7123543

shorten changelog

076c7de

Try to fix panning on OSX

6508758

u-pr Bot reviewed Sep 23, 2026

View reviewed changes

Comment thread com.unity.mobile.android-logcat/Editor/AndroidLogcatScreenCaptureWindow.cs Outdated

Comment thread com.unity.mobile.android-logcat/Editor/AndroidLogcatLiveStream.cs

todi1856 added 3 commits September 23, 2026 16:35

update screenshot

b2e6c2f

apply review fix

58fd75e

apply review fix

f1d921f

u-pr Bot reviewed Sep 23, 2026

View reviewed changes

Comment thread com.unity.mobile.android-logcat/Editor/AndroidLogcatCaptureScreenshot.cs

Comment thread com.unity.mobile.android-logcat/Editor/AndroidLogcatLiveStream.cs

Apply review fixes

c58dc0f

u-pr Bot reviewed Sep 23, 2026

View reviewed changes

Comment thread com.unity.mobile.android-logcat/Editor/AndroidLogcatLiveStream.cs

Track keys which were held

b5bf917

u-pr Bot reviewed Sep 23, 2026

View reviewed changes

Comment thread com.unity.mobile.android-logcat/Editor/AndroidLogcatScreenshotInfo.cs

todi1856 added 3 commits September 23, 2026 17:36

show live view when opening the window

68f3993

Decouple layout viewer from screen capture

638bbef

LayoutView->LayoutViewer

a280502

apply design changes

3243bee

todi1856 requested a review from IGuscin September 25, 2026 09:12

todi1856 added 21 commits September 25, 2026 14:58

Fix https://jira.unity3d.com/browse/UUM-154320 Openterminal was not w…

5572bf3

…orking, since UseShellExecute changed to false in CoreCLR for default value

reset protocol version to 1

cc39ffa

Remove Open/Save from screen capture

b8de601

Move Capture button to Take Screenshot

daa0dfe

fixate width dropdown Screenshot/Video

257a1e3

expand status bar

5b456e7

expand

2bcfb36

Report to status bar

cbbf432

Add setting for showing where screenshots are saved

501a95e

fix status bar style

901e3f8

add cog for quickly accessing settings

14e33a6

add multiselection to captures

fa688d0

Draw bounding box around screenshots/live

d2248ae

Copy To for multiselection

9bc14a9

add image icon

0b59277

fix status bar for video

07830bf

fix ui

8bc5901

Add fold control

fbed84a

move access to preferences to kebab icon

47c3c2a

Make a help box

32b0d14

folder to path

fa949b5

todi1856 mentioned this pull request Sep 30, 2026

Livestream server #226

Open

Conversation

todi1856 commented Sep 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Type of change:

Description

Improved screenshot window

Settings

Checklist for PR maker

Testing status

Devices:

Testing checklist

Uh oh!

todi1856 commented Sep 23, 2026

Uh oh!

u-pr Bot commented Sep 23, 2026

Uh oh!

u-pr Bot left a comment

Choose a reason for hiding this comment

💡 Harness Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

todi1856 commented Sep 23, 2026

Uh oh!

u-pr Bot left a comment

Choose a reason for hiding this comment

💡 Harness Review

Uh oh!

Uh oh!

Uh oh!

todi1856 commented Sep 23, 2026

Uh oh!

u-pr Bot left a comment

Choose a reason for hiding this comment

💡 Harness Review

Uh oh!

Uh oh!

Uh oh!

todi1856 commented Sep 23, 2026

Uh oh!

u-pr Bot left a comment

Choose a reason for hiding this comment

💡 Harness Review

Uh oh!

Uh oh!

todi1856 commented Sep 23, 2026

Uh oh!

u-pr Bot left a comment

Choose a reason for hiding this comment

💡 Harness Review

Uh oh!

Uh oh!

todi1856 commented Sep 24, 2026

Uh oh!

todi1856 commented Sep 29, 2026

Uh oh!

u-pr Bot commented Sep 29, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

todi1856 commented Sep 23, 2026 •

edited

Loading