chore(audience-sample): trim 30 unused packages and engine modules#766
Closed
ImmutableJeffrey wants to merge 19 commits intochore/audience-linux-perf-test-pollingfrom
Closed
chore(audience-sample): trim 30 unused packages and engine modules#766ImmutableJeffrey wants to merge 19 commits intochore/audience-linux-perf-test-pollingfrom
ImmutableJeffrey wants to merge 19 commits intochore/audience-linux-perf-test-pollingfrom
Conversation
… (SDK-317)
- Replaces game-ci/unity-test-runner@v4 with docker run of
unityci/editor:ubuntu-${unity}-linux-il2cpp-3, wrapped in xvfb-run.
- Without an X server every [UnityTest] returned inconclusive
(passed=0, failed=0). The action's USE_EXIT_CODE=false hid Unity's
exit code 2, dorny/test-reporter does not flag inconclusive, so
each Linux cell silently went green for ~3 min without executing
a single test. macOS by comparison ran 39 tests in ~25 s with all
39 passing.
- xvfb-run gives Unity a software-rendered virtual display via
mesa-llvmpipe; no GPU required.
- One image tag covers both backends (verified against the Mono cell
log from run 25492697422 which pulled the il2cpp tag).
- Per-run license activation + return mirrors the dropped self-hosted
Linux job (commit 2658686).
- Adds a post-step that parses playmode-results.xml via xmllint and
fails the cell when inconclusive > 0 or passed+failed == 0; stops
any future regression from silently re-greening the matrix.
- timeout-minutes: 30 since cells now run ~5-10 min of real work.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- xmllint is not actually preinstalled on ubuntu-latest-8-cores; the previous guard step failed with "command not found" while the underlying xvfb + Unity invocation had executed all 39 tests successfully (passed=39, failed=0, inconclusive=0). See run 25530011079. - Parses passed/failed/inconclusive off the root <test-run> line via grep instead. Unity's NUnit writer keeps the summary attributes on one line, so the parse is reliable and adds no apt deps. - Validated locally against the playmode-results.xml from the failing cell: extracts passed=39 failed=0 inconclusive=0 as expected. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- PlayMode tests on StandaloneLinux64 build a separate player binary that runs the suite. The editor stdout the workflow tees never sees the player's HTTP traces, OnError callbacks, or cert failures. - Without this capture, "39 passed" only tells us FlushAsync returned without throwing. Whether events reached CDP cannot be verified, and any silent OnError-class failure is invisible. - Adds a find + cp inside the docker bash that copies every Player.log under /root/.config/unity3d/<Company>/<Product>/ into artifacts/Player-<Company>-<Product>.log before the container exits. - upload-artifact now includes artifacts/Player-*.log alongside playmode.log and activation.log. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- OnSdkError now also calls Debug.LogError; SDK Log.Writer adapter now also calls Debug.Log / Debug.LogWarning. The in-app log pane keeps its existing behavior. - Without the mirror, OnError fires (HTTP / TLS cert / 4xx / 5xx) were visible only in the in-app pane, which disappears with the player process and is not captured by NUnit's test-results.xml. The StandaloneLinux64 cells passed all 39 tests but emitted zero SDK output to Player.log, so we cannot tell whether events actually reached CDP or whether OnError silently fired for every flush. - Mirroring to Debug.Log surfaces those entries in Player.log, which the workflow now uploads as artifacts/Player-Immutable-audience.log. - Sample app users also benefit: SDK warnings and onError now show up in the Unity console instead of being trapped in the UI panel. - Calls to Debug.Log are thread-safe in Unity, so the existing off-main-thread behavior of OnSdkError and RouteSdkLogToPane is unchanged. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…stall the cell (SDK-317) - Wraps `xvfb-run unity-editor ...` in `timeout --signal=TERM --kill-after=10 1320` inside the docker bash. SIGTERM at 22 min, hard SIGKILL 10s later if the editor still has not exited. - Unity 6 on Linux has a known shutdown hang: tests run, the runner writes playmode-results.xml, then the editor begins `Application is shutting down...` and never fully terminates - likely a leftover thread or a player process xvfb-run is still tied to. Without the cap, the cell sits idle until GitHub's 30-min job timeout and the upload-artifact step never runs, so the results XML and Player.log are lost. - 22 min covers Unity 6 IL2CPP build (~5 min) + 39-test suite (~10-15 min) with a 2-min buffer. The 8-min slack to the cell timeout lets the post-Unity steps (license return, Player.log capture, artifact upload) all complete. - Annotates exit codes 124 and 137 (timeout and post-grace SIGKILL) so the artifact reader can distinguish a real test failure from a shutdown-hang kill. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
… summary (SDK-317) - Adds a `RuntimeInitializeOnLoadMethod` in AudienceSample that prints `[CI] buildGuid=... runId=... cellId=...` to Debug.Log once per player startup. The line lands in Player.log, which the workflow already uploads as an artifact. - Workflow passes `AUDIENCE_TEST_RUN_ID` (github.run_id + run_attempt) and `AUDIENCE_TEST_CELL_ID` (target + backend + unity) as env vars through the docker container, where Application reads them via Environment.GetEnvironmentVariable. - Adds a post-step that greps the `[CI]` line from Player.log and writes it into `$GITHUB_STEP_SUMMARY`. The summary is visible from the Actions UI without downloading the artifact. - buildGuid is per-build and already lands on every game_launch event the SDK emits via DeviceCollector. Pairing the [CI] line with the same buildGuid on CDP gives a one-to-one match between a CI cell and the events it produced - removes the ambiguity left by the EPYC + Mesa llvmpipe + LinuxPlayer fingerprint, which can be shared by other Linux PR runs in the same window. - Local / production runs leave the env vars unset; the printed values are empty. No production behavior change. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Drops the `timeout 1320` wrapper. Replaces it with a watchdog loop
inside the docker bash that polls artifacts/playmode.log every 5 s
and signals Unity 30 s after "Test run completed" first appears.
- 40 min hard cap as a fallback for the case where Unity never logs
"Test run completed" (player hang, etc.); 15 s SIGKILL grace after
SIGTERM if the editor refuses to exit.
- Bumps cell timeout-minutes from 30 to 45 to cover the inner 40 min
cap plus post-Unity steps (license return, Player.log copy, artifact
upload, dorny/test-reporter).
- Why: a fixed timeout that fits Unity 2021.3 (~5-7 min cells) cuts
Unity 6 off mid-run; a fixed timeout sized for Unity 6 makes 2021.3
cells wait up to 30+ min on a shutdown hang they would not have
hit. The previous 22-min cap killed Unity 6 cells before tests
could finish writing playmode-results.xml. The watchdog adapts to
whatever the actual test runtime is, then catches the Unity 6
Linux shutdown hang ("Application is shutting down..." that never
completes) without waiting on it.
- Also captures `tail -F` of the log to job stdout while Unity is
alive, so the live build / test progress streams to the GitHub
Actions log as before.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…DK-341) - DiskStore.ReadBatch was the one path that called Directory.GetFiles(_queueDir) without the DirectoryNotFoundException guard the rest of the methods (DeleteAll, ApplyAnonymousDowngrade, TryDelete) already use. - Linux Mono / IL2CPP test cells run a SetUp that deletes the SDK persistent dir between every test. A background flush timer started by the prior test can fire after the delete; the resulting GetFiles raised DirectoryNotFoundException which propagated through HttpTransport.SendBatchAsync to OnError, made the SampleApp log "flush() Err" instead of "flush() Ok", and failed the test. - Same shape was hitting Shutdown's flush path: the AggregateException the caller saw was wrapping the same DirectoryNotFoundException. - Catch + return Array.Empty<string>(). Empty result is the correct semantics: a deleted queue dir has no events to send. Surfaced by run 25539153233 once SDK-317 (PR #754) landed real Linux PlayMode coverage. Affected 9 cells (2 Mono, 7 IL2CPP) on Unity 6. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…DK-317)
- New [UnityTest] AudienceCiTestMarker_EmitsRunMetadata. Skips
outside CI (AUDIENCE_TEST_RUN_ID and AUDIENCE_TEST_CELL_ID both
unset). Inside CI, calls LoadAndInit then emits one
ImmutableAudience.Track("audience_ci_test_marker", { source: "ci",
ciRunId, ciCellId }).
- Lets dashboards filter test traffic in / out:
- select event_name = "audience_ci_test_marker" lists every CI run
that hit CDP, with run / cell metadata directly on the row.
- properties.buildGuid on the marker matches the buildGuid the SDK
auto-emits on every game_launch event from the same cell binary,
so a follow-up filter pulls the rest of the cell's events.
- Standalone test rather than a hook in LoadAndInit so the marker
stays explicit and opt-in. Other tests do not silently emit a
side-event; the SampleApp and SDK carry no CI / QA-specific
knowledge.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- AUDIENCE_TEST_RUN_ID moves to workflow-level env (set once per workflow run, no matrix dependency). - AUDIENCE_TEST_CELL_ID moves to job-level env on the playmode (Windows + macOS self-hosted) and playmode-linux (GitHub-hosted) jobs. Job scope so matrix.target / matrix.backend / matrix.unity resolve. Both test jobs now share the same definition. - Drops the duplicated step-level env entries inside the Linux Run PlayMode tests step. Step env now only carries credentials and matrix.backend / matrix.unity passthroughs. - The Linux docker invocation already passes the vars through with --env AUDIENCE_TEST_RUN_ID --env AUDIENCE_TEST_CELL_ID; that line reads the values from the inherited shell env regardless of where they were originally set. - Result: every CI cell on every platform (Linux + Windows + macOS) emits AudienceCiTestMarker_EmitsRunMetadata with the correct ciRunId / ciCellId, defined in two places instead of three. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Temporary instrumentation. Logs to Player.log: - ctor initial count + dir + instance hash - every BumpCount delta + new value + stack trace + instance hash Goal: identify which call sites fire BumpCount(-1) without matching BumpCount(+1) in the StatusBar_QueueSizeIncrementsAfterTrack failing on Linux Unity 6 cells. Revert before merge. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…DK-340) - Drops xvfb screen from 1280x720 to 320x240 (12x fewer pixels per frame). Per-frame fill on mesa-llvmpipe is the dominant cost on Unity 6 Linux; smaller render target attacks the bottleneck directly. Tests never assert on pixel content, only on UI Toolkit log-row presence via VisualElement tree, so layout still works. - Adds -force-glcore -screen-fullscreen 0 -screen-width 320 -screen-height 240 to the Unity args. Unity 6 prefers Vulkan on Linux and falls back to OpenGL after a per-frame negotiation overhead. -force-glcore skips that, mirroring the path Unity 2021.3 takes by default. Explicit -screen-* matches the xvfb size so the player does not request a resize. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
WaitForCondition and WaitForLogEntry yielded once per Unity frame, which on Linux + Mesa llvmpipe under Unity 6 ran at 1 to 2 fps. Each predicate check therefore happened only once per second and a 30 second budget hit the cap routinely, dragging the 39-test suite from a few minutes into the half-hour band. Switching the yield to WaitForSecondsRealtime(0.05f) decouples polling from frame pacing. Tests check 20 times per second regardless of render rate, exit as soon as the condition holds, and frame-rate-bound players no longer artificially extend the wall time.
…ild time Adds GraphicsApisLinuxOverride, an editor build hook gated on the AUDIENCE_LINUX_GLCORE_ONLY env var. When the env is set and the build target is StandaloneLinux64, it pins PlayerSettings graphics APIs to OpenGLCore only, dropping Vulkan from the active list. The Linux container has no GPU and the player runs on Mesa llvmpipe via -force-glcore. Vulkan was active in the build only because Unity's default Standalone API list includes it; the shader compiler emitted both glcore and vulkan variants for every shader. On the Unity 6 cell this was 213 of 413 compiles wasted on a code path the player never hit. Wires the env var through the playmode-linux job's docker run so the override fires on Linux PlayMode CI builds. Local builds and other targets see no change because the env var is unset and the hook short circuits on non-Linux build targets.
The previous xvfb-run line passed -screen 0 320x240x24 in --server-args intending to shrink the virtual desktop to cut llvmpipe fragment-fill work. The player log on PR #764 (the deep profile capture) shows the xvfb desktop is 1280x1024 anyway: xvfb-run with -a auto-picks a display number and its own default screen geometry, and the supplied -screen override does not always take effect. The flag did not change behaviour. Unity opens its own GL context at -screen-width by -screen-height (320 by 240, set on the editor command line below), so per-frame fragment fill is already at that resolution regardless of the xvfb desktop size. Drops the redundant -screen flag while keeping the X11 extension flags that do matter (-ac, +extension GLX, +render, -noreset). Updates the adjacent comment block to credit Unity's own -screen-width/-screen-height as the actual fill-rate control.
Commit e7718f6 added a comment block above the xvfb-run line that contained 'xvfb's own screen size'. The apostrophe terminated the outer bash -c '...' single-quoted heredoc, the rest of the inner script became outer-shell tokens, and every Linux PlayMode cell on PR #765 exited 1 right after license activation (cell time 90 seconds). Rewords the comment without the apostrophe. No semantic change to the docker invocation or the Unity command line.
…runs Adds PlayerProfilerLogger gated on UNITY_STANDALONE_LINUX and the AUDIENCE_PLAYER_PROFILE_PATH env var. At BeforeSceneLoad inside the test player process, points UnityEngine.Profiling.Profiler at the configured path and starts a binary log of every captured frame. PR #764 captured a deep profile of the editor process. The actual test loop runs in a separate PlayerWithTests subprocess that the editor launches and waits on. The editor profile only showed roughly 86 sec of editor activity over a 27 min cell, the rest being editor idle waiting on the player. This hook plugs that gap. Wires the env var through the playmode-linux job's docker run and adds player-profile.raw to the upload-artifact path so the capture is downloadable for offline analysis in Unity Editor. Note: enables regular profiling, not deep. Deep profiling on the player would need -deepprofiling on the player command line, which Unity Test Framework does not expose for editor-launched test players. Regular profiling still surfaces per-frame CPU and the function hot list, which is what we need to identify what is eating roughly 37 sec per test.
…pe rasterisation PR 765 captured a player profile that showed the Unity 6 Linux cells spending roughly 4.5 seconds per frame in Gfx.PresentFrame self time on the render thread. Camera.Render was 2 ms and UI.RenderOverlays 1.45 ms per frame; the 4.5 sec is llvmpipe walking the deferred command buffer and rasterising approximately 2920 batches and 7520 triangles per frame, almost all of which are UI Toolkit log rows generated by the SampleApp's accumulating log pane. LinuxLogPaneSuppression registers a SceneManager.sceneLoaded handler that sets the log ScrollView to display:none after each scene load. display:none skips layout and render but keeps elements in the visual tree, so HasLogEntry (which walks contentContainer.Children() and inspects userData by reference) still observes log rows correctly. Engages only on StandaloneLinux64 builds via #if UNITY_STANDALONE_LINUX. Mac and Windows PlayMode runs are unaffected.
…SampleApp Drops 30 of 39 manifest.json entries that the SampleApp does not use. Verified by grepping the SampleApp source, the SampleApp scene, and the Audience SDK runtime for any reference to each removed module or package. Removed packages: - com.unity.textmeshpro: no TMP_Text or TMPro references - com.unity.timeline: no PlayableDirector or TimelineAsset references - com.unity.ugui: SampleApp uses UI Toolkit, no Canvas references - com.unity.visualscripting: no ScriptMachine or graph references Removed engine modules: - ai, animation, assetbundle, cloth, director, imageconversion, particlesystem, physics, physics2d, screencapture, terrain, terrainphysics, tilemap, ui, umbra, unityanalytics, vehicles, video, vr, wind, xr: no runtime references in SampleApp or audience SDK - unitywebrequestassetbundle, unitywebrequestaudio, unitywebrequesttexture, unitywebrequestwww: SDK uses System.Net.Http.HttpClient, not UnityWebRequest variants Kept (verified used or required): - com.immutable.audience: the SDK - com.unity.test-framework: test runner - com.unity.modules.androidjni: Android plugin compatibility - com.unity.modules.audio: AudioListener present in SampleApp scene - com.unity.modules.imgui: test runner overlay - com.unity.modules.jsonserialize: defensive - com.unity.modules.uielements: UI Toolkit, primary UI surface - com.unity.modules.unitywebrequest: defensive (test framework) Expected impact: - Smaller IL2CPP and Mono player binaries. - Faster IL2CPP compile and Bee/Tundra build phases on every cell. - Faster project init (less to load, fewer InitializeOnLoad hooks). - Per-frame UI Toolkit cost on llvmpipe is unchanged; that is targeted separately by PR #765's log-pane suppression. Risk: a build link error on a platform that needs a removed module shows immediately in CI; revert is one-line.
ImmutableJeffrey
force-pushed
the
chore/audience-trim-unused-deps
branch
from
2749f3c to
1834c9b
Compare
ImmutableJeffrey
changed the base branch from
chore/sdk-318-linux-playmode-xvfb
to
chore/audience-linux-perf-test-polling
ImmutableJeffrey
force-pushed
the
chore/audience-linux-perf-test-polling
branch
from
153b0f8 to
cf5e5d9
Compare
Collaborator
Author
|
Superseded by #765 (the manifest trim from this PR is included in the squash). |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Drops 30 of 39 dependencies from `examples/audience/Packages/manifest.json` that the SampleApp does not use. Verified by grepping SampleApp source, scene, and audience SDK runtime for any reference to each removed entry.
What's removed
Packages: textmeshpro, timeline, ugui, visualscripting.
Engine modules: ai, animation, assetbundle, cloth, director, imageconversion, particlesystem, physics, physics2d, screencapture, terrain, terrainphysics, tilemap, ui, umbra, unityanalytics, vehicles, video, vr, wind, xr, unitywebrequestassetbundle, unitywebrequestaudio, unitywebrequesttexture, unitywebrequestwww.
What's kept
`com.immutable.audience`, `com.unity.test-framework`, plus engine modules: androidjni, audio, imgui, jsonserialize, uielements, unitywebrequest.
Why
Default Unity project templates include many packages and modules that a focused audience-tracking SampleApp does not touch. Each unused dependency contributes IL2CPP compile time, Bee/Tundra build phases, project init time, and player binary size on every CI cell across every platform.
This is a separate concern from per-frame UI Toolkit cost on Unity 6 Linux; the log-pane suppression on PR #765 targets that.
Risk and rollback
A build link error on a platform that needs a removed module shows immediately in CI. Revert is one line in this PR. Stack-style on chore/sdk-318-linux-playmode-xvfb so the diff is just the manifest change.
Test plan
🤖 Generated with Claude Code