Add screen_capture_format attribute to ios_xctestrun_runner - #3021
Conversation
72162ec to
8501095
Compare
8501095 to
73046fb
Compare
|
lgtm, can you rebase |
Threads a `PreferredScreenCaptureFormat` value (added in Xcode 15) into the generated xctestrun file so UI tests can capture a full screen recording instead of just screenshots. Accepted values are `""` (default; keep Xcode's platform default), `"screenshots"`, and `"screenRecording"`. Setting a non-empty value implies use of `xcodebuild test-without-building` (the direct `xctest` path does not read the xctestrun file) and implies `create_xcresult_bundle = True` (the recording is only observable in the `.xcresult`). Screen recordings are system attachments, so `attachment_lifetime` must be `"keepAlways"` or `"deleteOnSuccess"` for the recording to survive into the `.xcresult`; this is documented on the attribute.
Review feedback: - Rename the attribute (and --xctestrun_* flag) to screen_capture_format: it is a passthrough for Xcode's PreferredScreenCaptureFormat key, and screen_recording = "screenshots" read as a contradiction. - Emit a runtime warning when a capture format is requested while attachment_lifetime is "keepNever", since the capture would be silently discarded before reaching the .xcresult bundle. - Drop the screenshots-value test: it exercised the identical code path as the screenRecording test and the attr values list already guarantees the value threads through. - Restore the pre-existing attachment-lifetime comment so the diff stays purely additive.
Review feedback: screen captures are system attachments, so with the default attachment_lifetime of "keepNever" the capture would be silently discarded after paying for the xcodebuild path and an xcresult bundle. Fail fast with a clear error instead of warning, and cover the failure in the runner test suite. The simulator serialization commits moved to their own PR (simulator pool), also per review feedback.
73046fb to
bdd8b97
Compare
|
Rebased onto main — the conflict was with CI note, correcting my earlier read of this: I first attributed this to the simulator-concurrency flakiness described in #3026, but that is at most part of the story and I overstated it. #3026 itself fails What the red ones have in common is that they invalidate the cached results of the simulator tests, so those tests actually execute instead of returning as cache hits — which is the condition #3026 addresses. But that is a correlation across PR contents, not a diagnosis: I have not been able to read the Buildkite logs to confirm which test actually fails, so I would not want the concurrency explanation taken as established. Happy to re-run, or to narrow this down if someone with Buildkite access can share the failing output. |
Summary
Adds a new
screen_capture_formatstring attribute toios_xctestrun_runnerthat threads Xcode'sPreferredScreenCaptureFormatxctestrun key (added in Xcode 15) into the generated file. Accepted values:""(default): key is not written, so Xcode's platform default is used — no behavior change for existing users."screenshots": capture screenshots only."screenRecording": capture a full screen recording of each UI test run, attached to the resulting.xcresultbundle.The same value can be set ad hoc for a single run without editing the runner target via
--test_arg=--xctestrun_screen_capture_format=<value>.Setting a non-empty value implies:
xcodebuild test-without-building(the directxctestpath does not read the xctestrun file).create_xcresult_bundle = True(the capture is only observable in the.xcresult).Both are documented on the attribute and echoed as
note:lines when triggered, matching the existingattachment_lifetime/xcodebuild_argsconventions.Interaction with
attachment_lifetimeScreen captures are system attachments, so
attachment_lifetimemust be"keepAlways"or"deleteOnSuccess"for the capture to survive into the.xcresult— the default"keepNever"discards it. This is called out in the attribute doc, and the runner emits awarning:at runtime when a capture format is requested while the lifetime is"keepNever".Test plan
//test:ios_xctestrun_runner_ui_test— 2 new template-emission cases:test_ios_ui_test_default_screen_capture_format_arg— assertsPreferredScreenCaptureFormatis not emitted when the flag is unset.test_ios_ui_test_screen_capture_format_arg— asserts the key +screenRecordingvalue +SystemAttachmentLifetime=keepAlwaysare all written when both flags are set.//doc:check_iospasses with regenerateddoc/rules-ios.md.buildifier --lint=warnpasses clean..xcresultproduced and opens in Xcode with the.movvideo attachment visible in the Attachments panel.test_ios_ui_test_passandtest_ios_ui_test_attachment_lifetime_argpass.